Tag Archives: Migration

Migration

This article covers how to import publishing pages to SharePoint.

This can help you if you need to bulk create pages in SharePoint and you have the data stored in a database or spreadsheet or if you need to migrate pages from another content management system such as WordPress.

yourpages

Scenario

We are going to assume that you have your page data in a database.  For this example we are going to use MySQL and the WordPress schema.  As you will see in the coming steps this could just as easily be any OleDB or ODBC data source and any schema.

Import Tool

We will use the free Import for SharePoint toolset to import the pages.

You download it from here.

When you download and install the import tool you will have full documentation and additional example import configuration files which will help further.

The Source

Column

The source data must contain a column with the HTML mark-up in it.

In our example content this is called ‘PageContent’.  Inside it looks a bit like this.

Select Statement

Now from the database source we need to ‘Select’ the data that will create our pages.  Since we are working with WordPress in this example the data is in wp_posts as we see below.

This select statement is also, cleverly, giving us the destination page name and setting up the page to be automatically published.

Import Configuration File

This is the file that tells Import for SharePoint how to create the pages in SharePoint.

The schema is fully explained in the documentation but the important bits for this exercise are explained here;

DestinationItemType

So we want to create publishing pages (We could alternatively create wiki pages, modern SharePoint pages, site pages or blog posts) but lets stick to the most common (publishing pages) for now.

<DestinationItemType>PublishingPage</DestinationItemType>

PageLayoutASPXName

So if your source select statement does not have a column of this name then “ArticleLeft.aspx” will be used.  If you want to use another page layout then ensure you select statement returns a column of this name containing the name of your desired page layout.

<PageLayoutASPXName>PageLayoutASPXName</PageLayoutASPXName>

ImportMapping

This bit maps your HTML data (in the column PageContent) to the SharePoint page content (field Page Content).

<ImportMapping xsi:type=”ImportMapping_String”>
<DestinationField>Page Content</DestinationField>
<SourceColumn>PageContent</SourceColumn>
</ImportMapping>

Execution

Ok so rather than re-invent the wheel we’ll let you read the documentation installed with Import for SharePoint on this one.

Result

Ok so originally in WordPress the page looked like this.

source

And now in out of the box SharePoint it looks like this.

result

Great, but seems a bit simplistic

Ok so we have shown how to import publishing pages into SharePoint.

Realistically a project is always going to be more complicated than that.

So lets talk about real life….

Targeting Branded SharePoint

Page Layout

So the destination is likely to be branded?  That’s no problem we’ve already talked about PageLayoutASPXName and custom branded SharePoint really just means using a different page layout.

Content Type Fields

But the destination page has extra fields, like managed meta data “Tags”, a Byline, an Article Date?  Again no problem you just need more of these ImportMappings to map data from your source into those additional SharePoint fields.

<ImportMapping xsi:type=”ImportMapping_String”>
<DestinationField>Title</DestinationField>
<SourceColumn>post_title</SourceColumn>
</ImportMapping>

Data Manipulation

So what if the source data is not in the exact format that SharePoint needs?

No problem this manipulation can be done in SQL as shown below.

WordPress was never going to contain a column giving us a file name like “MyPage.Aspx” so we create one on the fly using concat here.

If (when?) your manipulations get too complex for inclusion in the SQL statement (on the fly) you can directly manipulate the source table, just make sure you take precautions if the source data is used by anything else (like working from a copy).

So what does this get used for?

We have seen this approach used for the following;

  • Legacy Content Management System (CMS) migration.
  • Bulk creation of pages from Excel
  • Scan to Mark-Up / Republishing – Loading data that has been scanned and OCR’d into pages.
  • WordPress to SharePoint Migration
  • Drupal to SharePoint Migration
  • Joomla to SharePoint Migration
  • Custom Intranet to SharePoint Migration

Great, Makes more sense now but I’m still an bit unsure

No problem just get in touch.

Share

Does the world really need another post on upgrading to SharePoint 2016?

Perhaps, perhaps not.  One thing we definitely couldn’t find out there was a balanced post.  A quick google of Upgrade to SharePoint 2016 tends to capture a lot of articles at either extreme, from the “TechNet super technical” article at one end to the marketing “Hey why don’t you let us upgrade SharePoint for you” .  Not much in between, and we love a gap in the market don’t we!

So what does this post cover?

  • Who is this post aimed at ?
  • When would be a good time to upgrade?
  • Why Upgrade?
  • Why Not – Migrate to Cloud SharePoint?
  • How to Upgrade ?

Who is this post aimed at?

Organisations running earlier versions of SharePoint on-premises.  This could be SharePoint 2003,2007,2010, or 2013 – Foundation, Standard or Enterprise.

There is an upgrade path for each of these versions and editions each with its own technical and even licencing facets.  There is no SharePoint 2016 foundation for example.

When would be a good time to upgrade?

Starting now really.  The SharePoint “Trinity” of SharePoint 2016, SQL Server 2016 and Windows 2016 are now all available.

Why Upgrade?

New Features

There is quite a bit of new stuff in 2016.

https://technet.microsoft.com/en-us/library/mt346121(v=office.16).aspx

Development updates

New SharePoint Development techniques such as the SharePoint framework are planned for 2016, align it with cloud SharePoint but are not likely to be made available for earlier versions.

In short an upgrade will allow you to develop better solutions which will last longer and be more compatible.

Path to the cloud

The hybrid capabilities of SharePoint 2016 and development updates ease your eventual route to the cloud.

Support

There are a lot of production platforms out there reliant on Microsoft software which is already in extended support or even out of support.  It’s worth looking at what version of SharePoint, SQL Server and Windows Server you are running and familiarising yourself with the support / lifecycle position.

You can do this here;

https://support.microsoft.com/en-us/lifecycle/search

Why Not – Migrate to Cloud SharePoint?

Cloud vs. on-premises is a big discussion which we are not fully exploring here.

A core consideration in respect to SharePoint when you are an existing user is customisation.  Any server side customisation which you are using (and was the norm in earlier SharePoint versions) won’t work in the cloud without major revision.  Moving to SharePoint 2016 Server, however, does not carry this restriction.

How to Upgrade ?

Upgrade Plan

Yup definitely going to need one of those, and a good one to.

What to put in the plan?

Communication

Key stakeholders, users, IT teams – This is going to impact everyone involved in SharePoint.

Training

Use and administration of the platform is going to be different post upgrade.

Customisation

Customisation (solutions) will need to be loaded and working on the new version.

You are going to want to test any customisation but branding changes, specifically master page changes, are not going to work without modification with the SharePoint 2016 user experience.

3rd Party

Establish clarity on the effect of an upgrade on any 3rd party products.

Approach

In place upgrade or side-by-side?

Content

If content is being moved as is then the standard out of the box process will do this.

If content is being re-organised then this infers a custom upgrade process or tool will be used.

SharePoint 2016 will upgrade content from SharePoint 2013.  If you are upgrading from 2010 or 2007 you will need a temporary SharePoint server to carry out each upgrade you previous skipped.  Alternatively a custom upgrade process or tool could be used.

Downtime

Downtime will need to be communicated and planned in or mitigated.

The amount of downtime is often determined by the amount of and age (version) of content currently in SharePoint which in turn affects the upgrade process duration.

Service Applications

SharePoint service applications and their correspondent configuration and state databases will need to be rebuilt or upgraded.

Resourcing

Depending upon the complexity of your current implementation you may need to resource infrastructure (SharePoint, SQL, Server builds), development, testing, training, change management, project management and execution.

You might choose to out source, in source or do it yourself for either the whole project or a particular element.  For example you might wish to DIY the infrastructure and soft skills but outsource development updates to your original supplier.

Execution

Who does what when and in which order.

 

Share

This article covers how and why to do a considered file share import to SharePoint.  Creating a folder structure, meta data, importing files with content types and why it is important to do this.

Before you import your files you have hopefully prepared them and maybe have some Excel spreadsheets to import from.

No?  Check out our article on preparing for import.

Scenario

So we are going to stick with the staff folder scenario prescribed in the above article.  Your needs are likely to be different but you should be able to transfer the thinking and techniques demonstrated here.

Import Tool

We will use the Import for SharePoint toolset to import the files.

When you download the import tool you will have some Excel files, import configuration files, and screenshots of the content types which match this scenario.  Using these will make the next steps much easier.

 

Bulk Folder Creation

We need a folder for each employee in our scenario.

We assume that you have created a document library, attached a custom “Staff Folder” content type and to that some site columns.

Using the import tool we can create the folders from our spreadsheet.  The sheet is shown below.

excelfoldersource

From this the import tool will create a folder structure in your SharePoint library. Import meta data such as employee number is attached to each folder.

createdfolders

File Import

Now we can import our files into SharePoint.

We assume that you have created a document library, attached custom “Staff Document” and “Staff Disciplinary” content types and to those some site columns.

We can use the Excel spreadsheet as the import source.

excelfilesource

Once the import has processed this the files will have been imported into the correct locations and with the correct meta data set against each one.

uploadedfiles

Why did we do this?

Ok so now we have a good structure to support common requirements.

Retention

How so let us assume that HR want to delete Staff folders 20 years after staff have left the business.  We can add this retention policy onto the staff folder content type and for employees who have already left we have the date already set (See Bulk Folder Creation) .

retention

Ok, so usually it’s a bit more involved that this but you get the point.

Search

Adding meta-data gives us a better chance of an item showing up in search results and in the instance of managed meta data will give us access to refiners on the search results page.

5000 Item Per Folder Limit

Ok so we know it’s not a good idea to have more than 5000 items in a folder.  But doing our import as set out in this article you should be able to design a great structure that works inside this boundary even it your original folder structure did not.

So is this a packaged solution for Staff Records?

Well the reality is that the treatment of employment records will vary for each jurisdiction, can often complicated by different treatment for pension records,  and the SharePoint implementation will change dependent upon whether you have a HR system and how that works.

That said it’s a great demo scenario and hopefully demonstrated some techniques will can be applied in all areas of your work.

Share

Migration of file shares into SharePoint is neither a new nor unusual requirement.

That said it is often done without appropriate preparation resulting in a poor end result that does not meet the overall objectives of the organisation.

It is a common misconception that you can get the full value out of SharePoint simply by dragging and dropping your existing file structure.

In this article we will run through a simple preparation example to see how easily an improved outcome can be achieved.  The article is based around staff (employee) records but the principals apply equally to most profiles of data.

The article covers;

  • Scenario – Be clear on the profile of data that you are going migrate.
  • Why Migrate? – Make sure there is a case for migration.
  • Design and Implement the Destination – Migrated data needs somewhere to live.
  • Catalogue the File Share – Catalogue your file shares, get a deeper understand of what is there.
  • Prepare the Excel Spread Sheets – Use Excel to cleanse the data and supplement meta data.
  • Get User Input – Use Excel to garner use input, particular where the users might need to plug in additional meta data missing from the current source.
  • Preparation Complete – You should have a source suitable as input to a migration process.
  • Importing the files – How to get the files into SharePoint

Scenario

Our theoretical scenario mirrors what is so often encountered when dealing with file shares.

We have a basic file structure.  At the outer nodes the files are stored.  The “meta data” is inferred by the location of the file in this structure as shown below.

Scenario_HR

The file share is the only data source.  There is no Staff Database.  If there was then this might be handled differently with that database providing some of the data.

The example has you have now seen is for some Staff Records.  Such records need to be stored in a compliant manner and retention scheduling is key to ensure that we retain only the correct records for each staff member.  The requirement in this post is vastly simplified in comparison to most Staff Record scenarios but it serves to illustrate the concepts very well.

Why Migrate?

Be clear on why you are migrating the data into SharePoint before you start the migration process.  You may need to design the migration process to ensure that the desired benefits are achieved.

In our scenario the key drivers are;

  • Compliance – Specifically data retention scheduling.
  • Efficiency – Consolidation into SharePoint.
  • Efficiency – Ease of use.
  • Efficiency – Process automation.

Design and Implement the Destination

Before you execute migration you need to have a destination to migrate into.

This is key for two reasons;

  • Migrating a live file share which is being updated is harder to manage.
  • Until you have designed and implemented the destination you won’t necessarily know how to define the import sources, in this example the two Excel spreadsheets.

For our scenario we have implemented;

  • A record centre – /sites/Staff/Records.
  • A record library – Records
  • A top level folder “Staff”
  • A content type for each staff folder “Staff Folder” which has a date of leaving field and retention action set from that date.
  • A content type for each staff record “Staff Record” which has an employee number field.
  • A content type for each disciplinary record “Staff Disciplinary Record” which has an Disciplinary Date and retention action set from that date because these records are kept for a shorter period of time.

It is sometimes useful to catalogue the file shares as part of this design process.  This will give you an insight into the scenarios that your destination will need to cope with.  The file shares can then be re-catalogued for migration at a later date.

Catalogue the File Share

To catalogue the file share we can use a PowerShell script.

You can run the script either from a PC as a user with access to the file share OR from the server hosting the file share.  The script below will audit both the files in the file share and the directories, each to separate CSV files.

The CSV files will contain the basic information that is available from the file system.

Files

Directories

Tip: Try and use UNC paths instead of mapped drive letters when cataloguing file shares.

If you are unsure how to execute PowerShell scripts then pop “how to execute a powershell script” into your favourite search engine.

Prepare the Excel Spread Sheets

The CSV files can be imported to Excel and turned into a spreadsheet.

This will enable us to automate the population of meta data that is so important to the success of migration projects

Folders

In Excel we can easily populate some extra columns (shown here in green).

Here we are going to create an import source which will create a folder of content type “Staff Folder” for each staff member.  This is going to have the Date of Leaving (which will drive retention schedules) and the staff name as meta data.  The employee name and the employee number will be extracted from the folder name from the file system using an Excel formula.

Excel_HR_Folders

Files

In Excel we can easily populate some extra columns shown in green.

Here we are going to create an import source which will import all of the files for the Staff.  The employee number is extracted from the folder name which will tell us destination folder and the set some meta data against each document.  This is extracted from the folder path.

Excel_HR_Files

Get User Input

One of the strengths of Excel is that most users will have skills in using it.

What this enables us to do is use it to capture from the user base any additional data and permit the users to generally cleanse the data.

Tip: Give the users some guidance notes on how the Excel spreadsheets should be completed.

Folders

Here the user has completed the date of leaving field which in turn will allow SharePoint to manage the retention schedule accordingly.

Excel_HR_DateOffLeaving

Files

Here the user has spotted that one file is a disciplinary document.  They have therefore changed the disciplinary date and content type accordingly.

Excel_HR_DiscDate

Preparation Complete

Once preparation is complete you should have a set of Excel spreadsheets that you can use to drive your folder creation and file migration and which will work with most good SharePoint migration tools including our own.

This should be double checked and quality controlled before you commence the migration process but the core work is done.

Preparation NOT Complete!

Struggling?  The likelihood is that the problem is either in the quality of your source data or the design of your destination and these areas will need to be revisited.

The good news is that that you have uncovered the issues at the right time – BEFORE you migration large quantities of data.

Importing the Files

There is a follow-up article here.

 

Share

If you are storing digital / scanned images in SharePoint you are most likely storing them as Adobe PDFs.

PDF is a great format to use because it views nicely and it also can accommodate full text searching of text in the image (more on that another time).

If you are now looking to implement records management for those documents this infers a requirement to keep the images for a long time.

This is where PDF/A comes in.  It is a reduced feature set of PDF which is guaranteed to be viewable in years to come whereas perhaps a more fully featured PDF document might not be in a couple of decades time.

pdf-a1

If you are scanning your documents into SharePoint the scanning software should be able to generate PDF/A compliant images for you though there might be an additional cost for this.

If you are migrating scanned images into SharePoint then it is worthwhile ensuring that these are generated as PDF/A compliant.

Share