We are considering this issue resolved as we have had no new reports of problems in the last 12 hours.
Posted Oct 23, 2019 - 14:57 MDT
Update
The code level change we made seems as suggested by Amazon AWS appears to have mitigated the issue. However, Amazon AWS is still reporting that the issue is under investigation.
We are considering all system as operational at this point, however we will continue to monitor the situation and provide updates.
If you still experiencing issues please let us know.
Posted Oct 23, 2019 - 04:32 MDT
Update
We have deployed the code change as suggested by Amazon AWS. The change adds extra region information to all our Amazon S3 requests which according to Amazon will help mitigate the issue.
We still recommend switching to either the Cloudflare or Google DNS servers which are confirmed working and are generally faster and apparently are a lot more reliable than those provided by your ISP. For more info on Cloudflare's DNS service check out https://www.cloudflare.com/learning/dns/what-is-1.1.1.1/.
We will continue to provide updates as we get info from Amazon AWS.
Posted Oct 23, 2019 - 02:34 MDT
Update
We are deploying the code change suggested by Amazon AWS to help mitigate the issue.
Posted Oct 23, 2019 - 02:04 MDT
Update
Amazon AWS is continuing to implement fixes to mitigate the issue...
Amazon have also suggested a code change to help mitigate the problem. We are currently working on implementing and testing that change and hope to have it deployed within the hour.
Posted Oct 23, 2019 - 01:27 MDT
Update
Amazon AWS reports that there service teams are actively investigating the issue. Unfortunately due to the complexity of the issue they are unable to provide an ETA.
A minority of users report that uploads, download and previews are failing. Investigation suggests that the effected users are experiencing issues due to problems resolving Domain Name Server* (DNS) queries to Amazon S3 services (our hosting provider).
DNS is a core component of the internet and is usually provided and configured auto-magically by your ISP when you connect to internet. Unfortunately being such a low level part of the internet puts it completely outside of our control.
As a temporary work-around for the issue we recommend switching to either the Cloudflare or Google DNS servers which are confirmed working (and are generally faster than those provided by your ISP).
* DNS converts human readable host names (e.g. digitalpigeon-dp-us-west.s3.amazonaws.com) into a series of numbers that computers understand. Whenever your computer attempts to make a connection on the internet the first thing it does is contact your configured Domain Name Server and attempt to resolve its host name into an actual network address. If this step fails then, which is what's happening in this case, the connection will fail.
Posted Oct 22, 2019 - 22:11 MDT
This incident affected: Oceania (Oceania File Servers), USA West (USA West File Servers), South East Asia (South East Asia File Servers), and Europe (Europe File Servers).