Jump to content


recent database server crashes

database crash data loss crash database

  • Please log in to reply
21 replies to this topic

#1 Aaron

Aaron

    Big Dork

  • Admin
  • 17795 posts
  • Gender:Male
  • Location:Nebraska
  • Favorite RMNP Hike: Green Lake

Posted 11 February 2012 - 02:45 PM

For the past week to ten days I've been battling our database server. If you've been following the story line of this tragedy you'll know this is not the first time I've had to resort to restarting the forums from a backup of the database server. Unfortunately, this time I had to go back to a 13-day-old backup. It's a long story. But until I can resolve the root problem with confidence I strongly suggest you make a backup somehow of any posts that you wouldn't want lost.

For now, as I've said, I'm not trusting that I've figured out what is/was going on. For now, I've got an extra layer of redundancy going, and I plan on adding another redundant layer once I get some time. At this point in the day I must stop working on the forums woes. (I've been at it for six hours so far) When I get time again this weekend I'll implement my next redundant layer as well as deal with some issues due to the data loss, ex. some stomp shirt orders were lost so I'll have to verify with each person that ordered to make sure who ordered what and what their mailing address is - I shouldn't have lost more than two or three orders.

The classifieds system was lost, and perhaps more. This trouble has worn me down quite a bit. I appreciate your patience and how you've "suffered" through this crap very well. I apologize to those who've lost valuable posts. I wish I could have made it through this without that loss.

--Aaron
p.s. - There is a possibility that this is from an outside source "hitting" our server and spiking some resources on it in order to takes us down. It could also be a search engine going crazy indexing us or something stupid like that. And among all the possibilities, it could've been due to the "emergency" server shutdown that our host imposed on our server. That gave us no advance warning to shut down our databases before they shut us down. So that could have harmed our database and the backups that I'd tried to use since then (but not this backup I've used for today....that backup is from days before that "emergency" shutdown).

#2 Aaron

Aaron

    Big Dork

  • Admin
  • 17795 posts
  • Gender:Male
  • Location:Nebraska
  • Favorite RMNP Hike: Green Lake

Posted 11 February 2012 - 02:55 PM

A tip to recover any posts (if you get on it quickly) is to use google's cached version of the forums. Search for the topic title and rmnpforums. Then view google's cache of that. It could save you A TON OF REWRITING!!

--Aaron :ballcap:

#3 DeeCeeM

DeeCeeM

    Your Virtual Tour Guide

  • Legend
  • 9503 posts
  • Gender:Female
  • Location:Jackson, MS
  • Favorite RMNP Hike: All of them

Posted 11 February 2012 - 02:56 PM

So sorry, Aaron. We can't thank you enough for all your hard work keeping this place going. Losing a few posts is a small price for us to pay, considering how much effort you've put into this.

#4 mwilson

mwilson

    RMNP addict

  • +Donation Rewards
  • PipPipPipPipPipPip
  • 5529 posts
  • Gender:Female
  • Location:Milford, IA

Posted 11 February 2012 - 03:25 PM

Aaron, as always, thank you so very much to all of your work on the forums. We appreciate and are sorry in has been such a pain in the rear recently.

#5 Aaron

Aaron

    Big Dork

  • Admin
  • 17795 posts
  • Gender:Male
  • Location:Nebraska
  • Favorite RMNP Hike: Green Lake

Posted 13 February 2012 - 04:14 PM

I'm leaning towards a situation where our site(s) are being attacked. I've got a lot of work to do with limited time. You're a patient bunch and I am thankful for that. I hope this doesn't drag out too long for me figuring out for sure what is happening and how to stop it. If/when the site goes down again go to Dixie's website for info on the forums. Here is the link to her site - http://www.mountaindrama.com/

--Aaron :ballcap:

#6 HighPlainsMedic

HighPlainsMedic

    Keep your eyes fixed where the trail meets the sky.

  • +Donation Rewards
  • PipPipPipPipPip
  • 941 posts
  • Gender:Male
  • Location:Fleming Colorado
  • Favorite RMNP Hike: Can't choose

Posted 13 February 2012 - 04:36 PM

Hang in there.. I wished there was something I could do..

#7 foxfire

foxfire

    So this is how bears stay warm?

  • +Donation Rewards
  • PipPipPipPipPipPip
  • 4227 posts
  • Gender:Male
  • Location:Clarinda, Iowa 51632
  • Favorite RMNP Hike: Hallet Peak

Posted 13 February 2012 - 04:40 PM

Feel For Ya Man! Wish I Good Help As Well!

Fox :smile1: :catwink: :peace:

Same Ole - - :smile1: :catwink: :peace: - - For Ya!

#8 DeeCeeM

DeeCeeM

    Your Virtual Tour Guide

  • Legend
  • 9503 posts
  • Gender:Female
  • Location:Jackson, MS
  • Favorite RMNP Hike: All of them

Posted 13 February 2012 - 04:46 PM

Just a reminder - if you post something you want to make sure isn't lost, keep a backup copy so you can post it again easily.

Besides checking my website, as Aaron mentioned above, you can also email me at webmaster@mountaindrama.com in case of emergency.

#9 foxfire

foxfire

    So this is how bears stay warm?

  • +Donation Rewards
  • PipPipPipPipPipPip
  • 4227 posts
  • Gender:Male
  • Location:Clarinda, Iowa 51632
  • Favorite RMNP Hike: Hallet Peak

Posted 13 February 2012 - 05:01 PM

This is an annoyance to me but, just being me, I can't imagine Aaron's and the other Admin's frustration! I will just be patient! I am hoping for an easy and fast way out of problems for Aaron. Sure that it might not be either of those but I can hope! :smile1:

Thanks Dixie for all you do as well!

Fox :smile1: :catwink: :peace:

#10 HighPlainsMedic

HighPlainsMedic

    Keep your eyes fixed where the trail meets the sky.

  • +Donation Rewards
  • PipPipPipPipPip
  • 941 posts
  • Gender:Male
  • Location:Fleming Colorado
  • Favorite RMNP Hike: Can't choose

Posted 13 February 2012 - 06:45 PM

View PostDeeCeeM, on 13 February 2012 - 04:46 PM, said:

Just a reminder - if you post something you want to make sure isn't lost, keep a backup copy so you can post it again easily.

Do you remember losing a toy as a child and then finding it again some time later? it was like Christmas all over again, I did not keep back ups of some posts.. lost in the sands of time.. But I can resurect them and live it again... The most important thing is fixing the forum truck, so we can all go on a ride..

#11 John

John

    Forum Fanatic

  • +Donation Rewards
  • PipPipPipPipPipPipPip
  • 18430 posts
  • Gender:Male
  • Location:Joplin, MO (Elev. 1,005')
  • Favorite RMNP Hike: Glacier Gorge

Posted 13 February 2012 - 06:58 PM

If you identify the attacker, deploy Jen. She destroys everything.

#12 Allie

Allie

    other half of the set <><

  • +Donation Rewards
  • PipPipPipPipPipPip
  • 8871 posts
  • Gender:Female
  • Location:Kansas City MO
  • Favorite RMNP Hike: Wild Basin Area

Posted 14 February 2012 - 06:02 AM

Good grief, Aaron. As everyone else has said, thanks for all you do. :hug:

#13 Rhonda

Rhonda

    RMNP Addict

  • Moderator
  • 6879 posts
  • Gender:Female
  • Location:Alvarado, Texas, elev 764 ft.
  • Favorite RMNP Hike: Odessa Loop

Posted 14 February 2012 - 09:01 AM

One strange thing I have noticed lately, and it is happening right now when I checked, is that it shows 245 guests are online right now. I don't know if that is something someone is doing to hack the Forums or what, but I knew it was not right.

I feel so bad for you, Aaron. I know this is a major headache. I wish I could help.

#14 Aaron

Aaron

    Big Dork

  • Admin
  • 17795 posts
  • Gender:Male
  • Location:Nebraska
  • Favorite RMNP Hike: Green Lake

Posted 14 February 2012 - 09:35 AM

I stumbled onto what I hope was the problem yesterday evening. I believe I've fixed the problem. Hopefully that was the true problem because I don't want to have the hassle of being attacked.

@Rhonda - about the high number of guests. Most likely that's mostly search engine spiders crawling the forums. With the big difference in our site since they've last visited (data loss from crashes) they'll index our site(s) again like crazy to "catch up" with what they think are changes we've made.

--Aaron :ballcap:

#15 DeeCeeM

DeeCeeM

    Your Virtual Tour Guide

  • Legend
  • 9503 posts
  • Gender:Female
  • Location:Jackson, MS
  • Favorite RMNP Hike: All of them

Posted 14 February 2012 - 11:45 AM

Time will tell, but the answer you came up with makes a whole lot of sense to me. Way to perservere! :hifive:




1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users