Over the last few days, we've had issues with a series of short but frequent network micro-outages from OVH's vRack system, and we've invested in new equipment to ensure your service remains stable.

What this means is that - for a number of very short periods - our servers were unable to talk to each other, and this has had consequences resulting in lost transactions (A list of incident times are attached to the bottom of this post)

To be clear, 99.99% of transactions are unaffected, and we're working with OVH to try to get this issue resolved, but we consider this an unacceptable situation and we've taken action to combat this problem directly.

So, we've invested in two completely independent transaction processing servers designed to catch and recover transactions when our main cluster has an issue.

These servers have been chosen very carefully. They are hosted by a completely different company, in a different country (even a different continent), and they're routed to Second Life via a completely different path (xo.net rather than Level3).

What does this mean? Even if our servers are completely offline, even if our host is completely offline, even if the whole of Canada is offline, even if Second Life's connection to the world is disrupted, we'll still recover transactions.

What's more, any time a transaction is lost and "caught" by this new safety net, it will be flagged with us so we can investigate any issues directly, and will no longer have to depend on end customers providing the transactionID.

We're committed to an absolutely stellar level of service, nothing less will suffice - and we'll continue to invest and innovate to perfect that.

~Casper

P.S:

List of disruptions (note that each incident had a minor impact, but the frequency is alarming). If a start/end time is not provided then the incident lasted for less than a minute.

01/04/2018 08:12-08:19 BST (GMT +1)
01/04/2018 10:04-10:05 BST (GMT +1)
31/03/2018 19:50-19:57 BST (GMT+1)
31/03/2018 17:16 BST (GMT+1)
31/03/2018 16:42 BST (GMT+1)
31/03/2018 10:05 BST (GMT+1)
31/03/2018 08:19 - 08:27 BST (GMT+1)
30/03/2018 21:08 - 21:13 BST (GMT+1)
29/03/2018 19:41-19:48 BST (GMT+1)
29/03/2018 16:52 BST (GMT+1)
29/03/2018 16:35-16:39 BST (GMT+1)
29/03/2018 15:54 BST (GMT+1)
27/03/2018 20:02 BST (GMT+1)
27/03/2018 17:17 BST (GMT+1)
27/03/2018 17:17 BST (GMT+1)
27/03/2018 16:33 - 16:34 BST (GMT+1)
27/03/2018 16:22 - 16:23 BST (GMT+1)
27/03/2018 14:46 BST (GMT+1)
27/03/2018 14:29 BST (GMT+1)
27/03/2018 13:20 - 13:56 BST (GMT+1)