Unplanned Downtime – Oct 12

We just had about 1 1/2 hours of downtime. I’m very sorry about the inconvenience. There was a software problem on our load balancer and I’m looking for the cause of it.


Return to Discogs Blog
16 Comments
  • Nov 13,2008 at 05:40

    I wondered, as a layman on this stuff, does operating a mirror site (one in US / one in EU — for example) help to beat these problems?

    I presume this is much too costly for Discogs anyway, but thought I’d ask as I thought it was the norm for many medium to big sites…

  • Oct 17,2008 at 11:23

    Probably. I tracedrouted it and I have 15 hops (which is kinda alot) to discogs. The seven last (exept for discogs) are all integra.net and that is where the major lag comes from atleast from my point.

    I am on a pretty decent line. 10Mbit/10Mbit, one hop away from a 2*10Gbit back bone university line (which has far from filled its capacity). From my point the slownes has always been coming from outside the US (I am in Europe) and then within the US it is where it starts slowing down. It is like that on all major us sites here (like youtube and so on).

  • teo
    Oct 17,2008 at 10:57

    We haven’t had any server or network issues this week. And I checked our traffic graphs and there is no noticeable drop or anything unusual. So I’m guessing this is a network issue outside of Discogs.

  • nik
    Oct 17,2008 at 07:10

    Thanks for the replies.

    I am very sorry for the disruption. I was aware of issues this morning, but not with the main site. I did a bunch of traceroutes when it wasn’t working, and emailed this over to Kevin. I am unsure of the exact nature of the problem right now, and whether it was an internal Discogs thing, or lies externally, but we’ll see if we can get to the bottom of it.

  • Oct 17,2008 at 06:41

    [quote]
    Was the browser ‘spinning’ with a message bottom left like “transferring data from ?????.com”.
    [/quote]
    Most like that, nothing would load at all. Unfortunately the proxy here won’t allow me to do a proper tracert.

  • Oct 17,2008 at 06:40

    I can’t do a tracert discogs.com, every step just times out.

    The nature of the no-loading was most of the time a blank screen after a long time of “waiting for http://www.discogs.com“. A few times I managed to receive some unformatted text. It was the same problem with any page I tried to reach on the site; artists, releases, forums etc.

  • Oct 17,2008 at 06:38

    Ditto

  • nik
    Oct 17,2008 at 06:37

    If any of you can run a traceroute to Discogs from your computer, and forward the results to me, that would be great.

    What was the exact nature of the no loading? Did any of the page come up? Did you get an error message? Was the browser ‘spinning’ with a message bottom left like “transferring data from ?????.com”.

  • Oct 17,2008 at 06:36

    same here

  • Oct 17,2008 at 06:15

    same here, no access to any page of Discogs for the whole day until now.

  • Oct 17,2008 at 05:53

    Same here, the site has been unreachable all day until now.

  • nik
    Oct 17,2008 at 05:36

    Hi Zyron – can you let me know what pages you were trying to get to?

    Also, if you could run a traceroute to discogs.com from your computer, and PM me the result, that would be cool.

  • Oct 17,2008 at 05:22

    Today I’ve been unable to reach the site for the last 5 hours.

  • nik
    Oct 13,2008 at 02:59

    Hey Jooles,

    Downtime is not good for anybody, and I know for a fact that Kevin does everything he can to minimize any disruption. I don’t think the Oct 12 downtime had anything to do with any issues on Oct 11th. Upgrades to the system are always done live, as they are not uncommon, and to shut everything down whilst they were happening would cause much greater disruption. Glitches are not always experienced (some updates happen without anyone knowing), but occasionally, something might cause a problem.

  • Oct 13,2008 at 02:48

    What about the downtime on Oct 11th?

    From [url=http://www.discogs.com/help/forums/topic/172753]this thread[/url] (same one jimthing linked to), where I asked about downtime for about an hour or more around 02:00 CET on Oct 11th: [quote=nik]Kevin is doing some updates to the servers, sorry for any problems during this process.[/quote][quote=Scoz]perhaps while he’s fucking with the servers he might [b]disable submissions and edits etc[/b]. oh and [b]beforehand put a notice up to warn people when he’s going to be doing it[/b]. just an idea.[/quote]
    My thoughts exactly. Less swearing, but exact same sentiment! ;-)

    C’mon [u=nik] & [u=teo], you’ve got to get a better process in place for these downtimes, whether planned or unplanned. Your users all agree that we add to the database for the “common good”, but it all feels so worryingly unstable when an unannounced downtime happens, with no warning message or information until afterwards. Not to mention the inconvenience it must cause to all your fee-paying sellers in the Marketplace.

  • Oct 12,2008 at 20:59

Leave A Reply