blog.frederique.harmsze.nl my world of work and user experiences

September 30, 2013

Cloudy upgrade finished, here comes the sun

Filed under: Office365 — Tags: , — frederique @ 17:57

It took us a while to prepare for the upgrade of our SharePoint Online 2010 to 2013, but we made it! We dealt with the customizations, tested everything beforehand, planned  in detail, and then performed the upgrade over a weekend. Again, not all smooth sailing. But now we are surfing Wave15 with its shining new possibilities.

Develop wave15 versions of customizations and test everything

As I mentioned in a previous post, we had to develop Wave15 equivalents of customizations, in particular the masterpage, the styling , some site templates, and a couple of webparts.

We tested everything first in our development and then our testing environment. These are separate domains in the cloud, so they are environments that need to be upgraded in their own right. Fortunately, Microsoft had upgraded these for us before they upgraded the production environment.

The testing environment has a good copy of the portal, but it does not contain a copy of all team sites that populate the production environment. We had set up examples of business solutions that were configured in the front-end: workflows using standard sharepoint functionality and SharePoint Designer, special views including conditional formatting and structured forms set up in SharePoint Designer, managed metadata navigation, custom page lay-outs, publishing pages with text and images, content query web parts, note boards, video web parts,…

So to check what would happen to the real life sites, we tested evaluation copies of the production sites. By the way: these evaluation copies are only available for a month, which is not that much if you are busy and have a lot of sites to test…

Involve the business

In the environment of this multinational client, the intranet team is responsible for the framework, but team site owners are responsible for their own sites. They can use the standard SharePoint options to change the configuration of their sites. As they know their sites better than anyone else, they are most suited to check if their sites survive the upgrade. And in any case, 200 eyes see more than 2…

So over a month before the upgrade, we asked all site owners to check the evaluation copy of their site, pointing them to an article with the main differences that they should be aware of. We had taken the other users off the evaluation copies, to avoid confusing the audience at large. Only the site owners could test them, plus the colleagues they decided to involve.
In the week before the upgrade, we warned them that the sites would be unavailable during the upgrade weekend, asking them to discuss it with their users. And that we would need them to check their sites after the upgrade was finished.
We also notified all users two days before the upgrade.

We approached the owners of known “special attention” sites in person and with more emphasis: sites with complex business solutions, like custom workflows, and/or site that impact the business heavily, because they are used a lot in people’s daily work. We had meetings with the owners of sites that looked like they might break. And we made appointments with some key owners, that they would test their sites during the upgrade weekend, so that there would be no surprises in these important sites when the first users arrived Monday morning.

After the upgrade was finished, we notified all users and specifically all site owners. And again these “special attention” sites and their owners received that special attention. Quite a few users and owners needed additional explanations or had some issues. They could contact the intranet team for that, so that kept us busy for the week or two after the upgrade.

Plan the upgrade in detail

We wanted to reduce the risks as much as possible, so we planned the upgrade itself in full detail, with a lot of testing.

Earlier

  • Investigate the risks and mitigations: once you do the upgrade, there is no way to roll back to the old version.
    • Fall back scenario in case the first checks are disastrous: don’t push the upgrade button if we know it breaks the site collection, but reschedule for the next weekend.
    • In case the portal breaks, prepare a very basic “homepage” that would link to the key applications outside of the intranet. If the other site collections were still working, this page would also link to those team sites.
    • Microsoft was confident that the actual content could not get lost; as a last resort we could get it from a back-up they make.
    • Prepare e-mail messages to warn users of any disaster.
    • Export our own lists and libraries, that we use to manage this project, to get offline emergency copies.
  • Plan the upgrade tasks and set up lists and views to facilitate them: when to upgrade the site collections, deploy the customisations, test the result, communicate to stakeholders
  • Prioritised the testcases, to make sure we checked during the upgrade weekend everything that could cause major upheaval if it would not work on Monday morning: the homepage, reading news, the basic functionality of team sites. We had about 200 testcases to perform the tests systematically and we had seen in our test environment that it would be too much work for one weekend. The testcases with lower priority could be tested in the days after the upgrade weekend.
  • Plan the time: during the weekend, starting after office hours on Friday. We compromised with the Brazilian site owners, whose office hours ended after our starting time. We knew the upgrade and the subsequent tests would take time, and we wanted to have that time without interfering too much with the business.
  • Plan the upgrade order of the site collections: it does not matter much according to Microsoft.
    • The MySites need to be upgraded first, but they were already upgraded when Microsoft flipped the switch.
    • The Content Type Hub and the Search collections needed to be upgraded before the customizations could be rolled out, so they were scheduled first.
    • The key portal site collections and examples of collections of each team site type were scheduled next.
    • The site collections with key Brazilian sites were scheduled last.
  • Schedule of the participants, which team members, technical specialists, Microsoft contacts would be involved at what times and how to contact them.
  • Finish all tests and fixes on the test environment in time, so that everything is stable.
  • Communicate to the site owners and users, when they cannot use the intranet because of this scheduled maintenance, what we expect of them with regards to testing, and what they should do if they have questions or issues.
  • List all “special attention sites” and the appointments we made with which owners, with quick links to open them.
  • List all “special attention elements” that were broken in evaluation copies and that were not available in the test environment, so that we can check them more thoroughly (with quick links to examples of pages with such elements).

Earlier on day the upgrade was started

  • Check and clean up before the upgrade of key functionality. Just making sure the evolving platform hadn’t broken anything important before we started with the upgrade, so that we would not blame the upgrade for coincidental issues. Clean up hidden web parts on the homepage, to keep the upgrade of such an important page as simple as possible.
  • Set up tests like starting workflow instances, for which we wanted to double-check that a running workflow would keep running. For the one workflow solution that was broken in the evaluation copy, we exported all relevant information, to make sure nothing would get lost.

During the upgrade session

  • A temporary portal homepage switched on at 18:00 on Friday, informing any user who would still visit the intranet, that it was being upgraded.
  • Site collections upgraded and customizations deployed in batches, their status administered in the list of site collections
  • Upgraded site collections tested, first the basics, to see if the upgrade has “landed”, at all and then the high priority test cases.Administer the result in the lists.
  • Check the priorised “special attention sites” and inform the site owners, taking into account the appointments we made with particular site owners who would test during the weekend
  • Check the “special attention elements”

So how did it work out in that upgrade weekend?

Most of the upgrade went smoothly and as planned…. except for the very first step: when we clicked the upgrade button, the site collections got stuck in ‘waiting to upgrade’. It took us 24 hours to get beyond that first step. After that, we rolled on at top speed, so that were were finished before everybody’s Monday morning, even for the Australian users.

  • The first site collections we tried to upgrade, got stuck at ‘waiting to upgrade’. We have a couple of completely standard out-of-the-box team site collections, so we tried those after the problem arose, but no luck there either.

    Stuck at 'waiting for upgrade'

    Stuck at ‘waiting for upgrade’

  • We had to ask Microsoft Support to get these upgrades going. In the end, they solved the problem by upgrading the sites from the back-end: the same upgrade functionality, but started from a script instead of the regular button.
  • We had some issues getting through to the appropriate Microsoft support engineers, because of a mix-up with the Premier support service id-number. Note to self: next time, check the support quick reference card as thoroughly as SharePoint itself, so that we can fix any misunderstandings beforehand.

Once the upgrade started rolling, we saw the following:

  • The upgrade time of a site collection mostly depends on the number of sites it contains: mostly it takes less than half a minute per site. But that still adds up to hours, if you have almost 800 sites and subsites in almost 30 site collections.
  • When the upgrade starts on a site collection, it really cannot be reached for a minute or so. : the user get the message “Sorry, something went wrong”.

    Sorry, something went wrong...

    For a very short time at the beginning, the upgrading site is really unavailable

  • After that, you see the Microsoft masterpage appear and you can navigate through the upgraded site collection root. The pink bar at the top warns you “We’re doing work to improve the site. Please bear with us if you experience temporary delays or glitches”

    Upgrade in process

    During the upgrade: “We’re doing work to improve the site. Please bear with us if you experience temporary delays or glitches”

  • There you can monitor the upgrade status page: the same page where you clicked the upgrade button.
    When the upgrade is finished, you can see that here. You can also see how long the upgrade took and if there were any errors. Attached to this page is a log file of the upgrade and – if there were any errors – a log file of the errors.
    Note: we did not get any errors during the upgrade of the site collections. The customisations only resulted in warnings, that were included in the Log File.

    Upgrade completed successful

    Summary of the upgrade status

And after the site collections were upgraded, the customisations were rolled out, the high priority test cases performed, the key sites checked, the owners notified… everything nice and smooth. So in the end, all our preparations did pay off.

Ok, we are still working in the Cloud. But now I am not suggesting that this cloud could burst ; this is the cloud of the silver lining and even the sun-kissed, fluffy white cloud floating in the blue sky. Here comes the sun!

July 1, 2013

Update on our cloudy upgrade

Filed under: Office365,SharePoint — Tags: , — frederique @ 23:32

We are starting to surf wave15. Since my previous post, we are taking big steps towards the upgrade of the SharePoint Online, although it still is not all smooth sailing. A summary of what we have seen so far.

The process

Microsoft announcements

Microsoft sent us e-mails that got more and more specific:

  • Preliminary: A few months before the upgrade, when the timeline was not final yet, Microsoft announced the upgrade, sketching the procedure and providing links to further information. We did not think that their statement that we did not have to prepare anything would apply to our rather complex implementation, and we were right about that.

    Preliminary announcement

  • In 4 weeks: a month before the upgrade, we received an announcement that also explained how we could ask them to postpone our upgrade via the administration pages, and a button to try the upgrade with a small group of users.
  • The upgrade date: Two weeks before the Microsoft upgrade we received the exact date when the upgrade would start.
  • Confirmation: After the Microsoft upgrade was finished (the next day), we received a confirmation.

You can postpone once

If the proposed date does not suit you, you can postpone it for at least a month via a button on the portal administration pages. You can only postpone one time, then you have to go with the flow. We did not postpone, because we wanted to get the upgrade as soon as possible.

Preparations to the upgrade below decks

Although we were upgraded by the end of June, the underlying software was already upgraded to wave15 by the end of March, as I described in a previous post. This broke some functionality at that time, but fortunately Microsoft repaired it soon afterwards. The advantage was that these features were wave15-ready after that, so that they did not cause any new issues in the next steps of the upgrade.

Ready for upgrade: Microsoft flips the switch

Microsoft flipped the switch on our environment end of June. The environment was ready for upgrade from then on, but the upgrade was not implemented on the front-end. The site collection owner can do that themselves, per site collection. I am very happy with that, because now we can flip the front-end switch when we are really ready.

The site collection owners see a pink bar at the top of each page: “Experience all that SharePoint 2013 has to offer. Start now or Remind me later”. Innocent users, including site owners, do not see this message. Only site collection owner have permission to perform the upgrade and only they see the message. So there is no risk of an over-eager site owner accidentally upgrading anything.

Check the health of your sites

On the site collection administration page, there is an option ‘Site collection health checks’. This generates a report of the elements that could cause problems.

Evaluation copy

I am very happy with the option to Try a demo upgrade. That provides you with an evaluation copy of a site collection, where you can check what happens to sites in general and complex and important real-life sites in particular. It is a upgraded copy of the site collection, so it is not exactly the same as upgrading the site collection itself. But it does give you a good idea of the upgrade result.

Upgrade or try a demo

Note: These evaluation copies have the same permissions as the original sites in the collection. I do not want innocent users to start working accidentally on the evaluation copies, so I have taken them out of the copies of real life sites. Otherwise they could enter the site via the search result by mistake.

These evaluation site collections are not visible in the site collection overview of the portal administration. You can find them, once they have been created, via the same ‘try a demo’. But the address of the copy is easily guessed from the original url:
https://ourcompany.sharepoint.com/sites/collection/ becomes

https://ourcompany.sharepoint.com/sites/collection-eval/

The copies remain available for a months and then they are automatically deleted. They are not synchronized in any way to the original site after they have been created. So make very sure that you use them only for experimenting and testing and not for any ongoing work.

Note: I received some of the evaluation copies within the hour, others within the two day limit. But a bunch of them got stuck and I had to ask Microsoft to provision them via a service request. So I was very happy I had requested them at an early stage.

 

Evalation copy

Upgrade the site collections

The owners need to upgrade their site collections within three months after Microsoft has made the upgrade available, otherwise the front-end will be upgraded automatically. The owner just need to push one button ‘Upgrade this Site Collection’.

But if your collection needs any specific settings or support files to work properly, you will have to do those right after you push that button. That is why we wait until we are sure we have caught all key issues and we will communicate to the users that they will not be able to use the intranet for a couple of hours.

Microsoft recommends to upgrade the ’MySite’ first. And actually, they already set this in motion when they flipped their switch: we have the new MyProfile since the end of June, although the rest is  still the “old” version. Users who already had a MySite (we didn’t have many yet), each had to upgrade their own MySite – otherwise others could not see their profile and not open it via the people search.

Communication and training

The users will get new options, but they also need to know how the classics work in the new interface. We will explain, especially to the site owners, how it will work after the upgrade.

The result

We are testing the upgrade in a separate environment and in evaluation copies. Here we found that many things kept working properly. But not everything.

Custom master pages disappear

When we upgrade a site collection, our own masterpage dropped out and we fell back to a standaard Microsoft master page; Microsoft had already warned for it, as many things now work in a different way. So we created a new version of the master page, with associated styles and script

The old site templates cannot be used

Site templates created for SharePoint Online 2010 are invisible and unusable in SharePoint Online 2013. This applies to programmed templates as well as templates that were created using the option ‘save site as template’. So we are recreating those.

Some pages don’t work

Pages that are not stored properly in a library and pages that have some problem in the page layout don’t work anymore. So we have to save these pages separately. Fortunately we have seen this error only for a few kinds of pages.

Sorry, something went wrong

Most settings keep working

The site navigation, for example, and list settings like choice fields and versioning survive the upgrade. This also applies to the views in web parts on pages, even if they have been enriched in SharePoint Designer with conditional, for instance. We haven’t had problems with content types either after we had survived the issues in the preliminary preparations that caused us to republish a lot of custom content types.

It is the custom web part, as a species, that is endangered: some keep working, others don’t. We have also had a few issues with the presentation of standard content query web parts: groupings were no longer displayed in several columns, following the web part settings, but in one column.

Workflows keep working

We have several advanced workflows created in SharePoint Designer 2010. They already were unstable, but the “half-way upgraded” situation that we are in now, between Microsoft making the upgrade available and us implementing it on the front-end, seems to destabilize them even further. But apart from that, it looks like the workflows keep working.
In order to take advantage of the new possibilities, of the new workflow engine, I will have to rebuild the workflows. Unfortunately I cannot do that yet, because the 2013 workflow option has been disabled in SharePoint Designer, pending the upgrade.

 

So at his time, we are testing and fixing things, and starting to explain to the site owners how their sites will work. Working on it…

March 31, 2013

Cloudy upgrade

Filed under: Office365,SharePoint — Tags: , — frederique @ 17:58

We are working in the cloud, in SharePoint Online. And our cloud is about to be upgraded to wave 15, i.e. ‘SharePoint 2013 Online’. We have not heard an exact date yet, but rumour has it that we’ll be upgraded in June.

This system of cloud upgrades is new to me. In other cases, we had to get into the server and do everything ourselves. Now Microsoft does all the upgrade work. And we need to find out what we should do to move along with it:

  • Test if all our configurations keep working or if anything breaks
  • List the changes in the interface and functionality that we’ll need to explain to the site owners or other users
  • Check which spiffy and useful new features we want to promote to our users
  • Prepare our communication and update our help content.

This week, we have already seen the first stages of that upgrade: below decks, the installation is already version 15. At least, when I check https://ourcompanyname.sharepoint.com/_vti_pvt/service.cnf
I see vti_extenderversion:SR|15.0.0.4454

Unfortunately, this change did cause some problems on the front end:

  • The Note Board does not work anymore
    My old comments are still visible in My Profile, but they are not displayed with the News article or Team Site home page where I had entered the comment in the Note Board
  • Metadata have dropped out of some places
    Some Content Query Web Parts have lost the metadata in the Presentation section.  And our own page layouts have lost the metadata in the property fields.
  • I cannot edit SharePoint Designer workflows anymore.
    My SharePoint Designer still works for things like conditional formatting, but I can an error message when I try to edit a workflow. I should move to SharePoint Designer 2013 for that, even months before the official upgrade.
  • MyQuickLinks.aspx has disappeared
    We use the old MyQuickLinks to manage our team site favourites, but now we get a ‘page cannot be found’ error.

Over the coming days, we’ll see if Microsoft will tweak the system or if we can do something to fix these problems.

In any cases, if you are managing a SharePoint Online environment, I recommend you keep an eye on it. We will definitely do that.

Powered by WordPress