Unplanned Downtime – Oct 12
We just had about 1 1/2 hours of downtime. I’m very sorry about the inconvenience. There was a software problem on our load balancer and I’m looking for the cause of it.
We just had about 1 1/2 hours of downtime. I’m very sorry about the inconvenience. There was a software problem on our load balancer and I’m looking for the cause of it.
You must be logged in to post a comment.
I wondered, as a layman on this stuff, does operating a mirror site (one in US / one in EU — for example) help to beat these problems?
I presume this is much too costly for Discogs anyway, but thought I’d ask as I thought it was the norm for many medium to big sites…
Probably. I tracedrouted it and I have 15 hops (which is kinda alot) to discogs. The seven last (exept for discogs) are all integra.net and that is where the major lag comes from atleast from my point.
I am on a pretty decent line. 10Mbit/10Mbit, one hop away from a 2*10Gbit back bone university line (which has far from filled its capacity). From my point the slownes has always been coming from outside the US (I am in Europe) and then within the US it is where it starts slowing down. It is like that on all major us sites here (like youtube and so on).
We haven’t had any server or network issues this week. And I checked our traffic graphs and there is no noticeable drop or anything unusual. So I’m guessing this is a network issue outside of Discogs.
Thanks for the replies.
I am very sorry for the disruption. I was aware of issues this morning, but not with the main site. I did a bunch of traceroutes when it wasn’t working, and emailed this over to Kevin. I am unsure of the exact nature of the problem right now, and whether it was an internal Discogs thing, or lies externally, but we’ll see if we can get to the bottom of it.
[quote]
Was the browser ‘spinning’ with a message bottom left like “transferring data from ?????.com”.
[/quote]
Most like that, nothing would load at all. Unfortunately the proxy here won’t allow me to do a proper tracert.
I can’t do a tracert discogs.com, every step just times out.
The nature of the no-loading was most of the time a blank screen after a long time of “waiting for http://www.discogs.com“. A few times I managed to receive some unformatted text. It was the same problem with any page I tried to reach on the site; artists, releases, forums etc.
Ditto
If any of you can run a traceroute to Discogs from your computer, and forward the results to me, that would be great.
What was the exact nature of the no loading? Did any of the page come up? Did you get an error message? Was the browser ‘spinning’ with a message bottom left like “transferring data from ?????.com”.
same here
same here, no access to any page of Discogs for the whole day until now.
Same here, the site has been unreachable all day until now.
Hi Zyron – can you let me know what pages you were trying to get to?
Also, if you could run a traceroute to discogs.com from your computer, and PM me the result, that would be cool.
Today I’ve been unable to reach the site for the last 5 hours.
Hey Jooles,
Downtime is not good for anybody, and I know for a fact that Kevin does everything he can to minimize any disruption. I don’t think the Oct 12 downtime had anything to do with any issues on Oct 11th. Upgrades to the system are always done live, as they are not uncommon, and to shut everything down whilst they were happening would cause much greater disruption. Glitches are not always experienced (some updates happen without anyone knowing), but occasionally, something might cause a problem.
What about the downtime on Oct 11th?
From [url=http://www.discogs.com/help/forums/topic/172753]this thread[/url] (same one jimthing linked to), where I asked about downtime for about an hour or more around 02:00 CET on Oct 11th: [quote=nik]Kevin is doing some updates to the servers, sorry for any problems during this process.[/quote][quote=Scoz]perhaps while he’s fucking with the servers he might [b]disable submissions and edits etc[/b]. oh and [b]beforehand put a notice up to warn people when he’s going to be doing it[/b]. just an idea.[/quote]
My thoughts exactly. Less swearing, but exact same sentiment! ;-)
C’mon [u=nik] & [u=teo], you’ve got to get a better process in place for these downtimes, whether planned or unplanned. Your users all agree that we add to the database for the “common good”, but it all feels so worryingly unstable when an unannounced downtime happens, with no warning message or information until afterwards. Not to mention the inconvenience it must cause to all your fee-paying sellers in the Marketplace.
See here: http://www.discogs.com/help/forums/topic/172753#2205838