ixwebhosting受攻击,官方解释,可是怎么跑出去,还是打不开

Yorgol

Incident Description:
Customers are experiencing intermittent slowness, poor connectivity, unresolved name servers, and FTP denial. This is caused by a DDoS attack. (Denial of Service) Our system administration department is blocking the offending IP addresses now. This attack is very large. After several attempts to isolate the source we had to temporarily block a large IP range from China, which is the source of the attack. This is a temporary block.

•12:02pm  Update:  Ok, we have a fix in (phew!).  We are hopeful and monitoring at this point.  Its not the type of situation like a hardware or software problem that we can say ‘fixed’ and its done, we have to watch a while and make sure there are no other cracks or holes that the attackers can get through.  Please try your sites by IP and DNS and our support folks are standing by if you still need help.  (our network engineers have collapsed and may be face down on their desks, though).  Lisa Grice – Director of Customer Service
•11:06am  Update:  Just a clarification point, this is mostly affecting cps: 8,9,&10  (which for those of you that are curious is why ixwebhosting.com is still up–we are in the same datacenter, but not the same CPs).  Also, we are making some progress, so some sites are coming back.  Please also check your site by IP, if IP works, then DNS caching will just have to clear for you to see your sites.
Which Customers are Impacted?
Customers with accounts located on Control panels 8, 9, and 10.
How are Customers Impacted?
Domain name not resolving, Site Slowness, Poor FTP connectivity. .How often will we be updated?
Hourly Time to Resolution (ETA)
unknown

Yorgol

Incident Updates
•- We are blocking offending IP addresses now.
•- 6am Update: After several attempts to isolate the source we had to temporarily block a large IP range from China, which is the source of the attack.
•- 7am Update: We are still working to resolve the DDoS attack on our servers. This attack has resulted in several IP ranges being blocked.
•-8:30am Update:  Unfortunately, we have no new information to share at this time.  A large amount of incoming IP addresses from one of our providers had to be blocked.
•10:04am  Update  At this time this is not an average DDoS attack, this is a large scale and professional attack from a large network with very high resources.  We are having to block almost all of Asia IP addresses, as this targeted attack is very sophisticated and is rerouting as we block individual IPs.  This is a seriously difficult attack, and all network and system engineers are working to restore the issue.  Currently the attack is targeting our DNS servers, and even adjusting as we move IPs and reroute them.  This is why some site are becoming available by IP but not DNS.  Also some customers will have DNS Caching that will have to clear or expire before sites are visible.  We apologize for the problems and promise we are doing all we can.  Lisa Grice - Director of Customer Service
•11:06am  Update:  Just a clarification point, this is mostly affecting cps: 8,9,&10  (which for those of you that are curious is why ixwebhosting.com is still up--we are in the same datacenter, but not the same CPs).  Also, we are making some progress, so some sites are coming back.  Please also check your site by IP, if IP works, then DNS caching will just have to clear for you to see your sites.
•12:02pm  Update:  Ok, we have a fix in (phew!).  We are hopeful and monitoring at this point.  Its not the type of situation like a hardware or software problem that we can say 'fixed' and its done, we have to watch a while and make sure there are no other cracks or holes that the attackers can get through.  Please try your sites by IP and DNS and our support folks are standing by if you still need help.  (our network engineers have collapsed and may be face down on their desks, though).  Lisa Grice - Director of Customer Service
•1:30pm Update: We have made some networking changes for our Name Servers.  We have also taken steps to improve the network connection to those servers.  We are still experiencing some problems and are continuing to work on it and monitor services.
•2:44pm Update: As of around 2:00 PM EST the DoS attack has ended.  There are still some customers located in China that will be unable to view their sites at this time.
Resolution Description
Blocking offending IP addresses as they arise.

Yorgol

posted in Other Issues by trevor

Follow comments via the RSS Feed | Leave a comment | Trackback URL
251 Comments to "Massive DDoS Attack – Resolved/Monitoring"
Thu, October 7th, 2010Enalotto says:

Any problems with server, sites was down about 1hour, now workingfor me, but not work for my partner located in other city, please fix it as quickly as possible

Link | Posted at 4:41 am EST

Thu, October 7th, 2010Kathy says:

I try PING the domain, I can not find the name server?

Link | Posted at 4:59 am EST

Thu, October 7th, 2010Darren says:

Are you sure you blocked only china, we are having problems in Europe reaching our website and email.

Link | Posted at 6:15 am EST

Thu, October 7th, 2010sushil says:

my site is down and website is unreachable

Link | Posted at 6:30 am EST

Thu, October 7th, 2010Naomi says:

Oddly enough, I can reach one of my subdomains but nothing else. It keeps telling me the server can’t be found. I hope you’ll be able to resolve this quickly.

Link | Posted at 6:38 am EST

Thu, October 7th, 2010Anne says:

What is the estimated down time?

Link | Posted at 6:40 am EST

Thu, October 7th, 2010GARY says:

What is the estimated downtime?

Why were we not informed earlier?

Link | Posted at 6:43 am EST

Thu, October 7th, 2010Nathan Goss says:

Why do you always state you will provide updates every 30mins, but go silent for well beyond that?

Give us an update, please.

Link | Posted at 6:47 am EST

Thu, October 7th, 2010Adnan says:

All of my sites are down why so ? I am doing PPC. You guys should have alternate servers

Try to fix it ASAP

Link | Posted at 6:55 am EST

Thu, October 7th, 2010Nathan Goss says:

When you update this page, stick an update time on there, so we can tell what is going on, and when

Link | Posted at 6:57 am EST

Thu, October 7th, 2010Cassius Krendler says:

Same question here. Why the 30mins update claim, if that’s clearly not happening. We need an ETA!

Also, I have had so many hack attacks on my IX account in the last 2 years… despite frequent password changes, firewalls etc. Is it possible that your server security is not up to scratch?

Link | Posted at 6:57 am EST

Thu, October 7th, 2010Nathan Goss says:

I’ll echo Cassius’ point – I too have had my site hacked on numerous occasions. Each time, I get the old “your site is not secure line” to lay the blame with me.

Each time I have had the issue, the queues for the Live Chat are HUGE – clearly, I wasn’t the ONLY person who had been compromised at that time.

Everything points back to the health and security of your servers.

Link | Posted at 7:02 am EST

Thu, October 7th, 2010cntotebags says:

Hello,I try PING the domain,appear:The address is invalid.why i can’t open my site domain?

Link | Posted at 7:02 am EST

Thu, October 7th, 2010Nathan Goss says:

Finally, somebody is listening!

Thanks for making it seem a *little* bit more like you are doing something and keeping us updated.

Link | Posted at 7:04 am EST

Thu, October 7th, 2010Nathan Goss says:

My site is currently inaccessible from the following locations:

Stockholm, Sweden
Montreal, Canada
London, UK
Dallas 4, TX
Herndon, VA
Houston 3, TX
Amsterdam 2, Netherlands
London 2, UK
Dallas 5, TX
Dallas 6, TX
Los Angeles, CA
Frankfurt, Germany
Atlanta, Georgia
New York, NY
Chicago, IL
Copenhagen, Denmark
Tampa, Florida
Seattle, WA
Washington, DC
Madrid, Spain
Las Vegas, NV
Denver, CO
San Francisco, CA
Paris, France
Manchester, UK

It appears you are blocking more than just a few IP blocks. It appears that you are blocking EVERYTHING!

Link | Posted at 7:06 am EST

Thu, October 7th, 2010Dave G says:

C’Mon IX. This is not acceptable. All of our websites have been unreachable now for over 4 hours. That’s OK if you are in the USA where most people are still in bed. Not acceptable for European Clients!!!!
DDoS is nothing new. If you claim to have the 99.9% Uptime “best support in the industry”, you would have a resilient failover system in place and be able to deal with DDoS.
Or is it primarily that you don’t have enough support people out of USA Working Hours?
We can’t afford this. Wake some people up !!!

Link | Posted at 7:07 am EST

Thu, October 7th, 2010Jeroen van de Peppel says:

Can’t reach my site from Australia either. Looking forward to your updates and prompt resolution

Link | Posted at 7:15 am EST

Thu, October 7th, 2010shojke says:

This is too much. For five hours my domains are not on-line. All morning I’m having calls from my clients. They are rely angry, and they don’t by my excuses about some China attack, or whatever. I will think twice about your services and my work with you.

Link | Posted at 7:16 am EST

Thu, October 7th, 2010Orna says:

Also from Cyprus. Very unprofessional. happened too many times.

Link | Posted at 7:17 am EST

Thu, October 7th, 2010Ron says:

It’s been 4 hours since YOU say the DDoS started, but no updates? You say you’ll give us hourly updates, but NOTHING?

And, people, it is an ATTACK, they couldn’t inform us sooner, attacks are USUALLY a surprise! And YES, while the attack is going on you can’t reach YOUR site! That’s the POINT of the attack! It clogs the pipes so nobody can reach your site.

Link | Posted at 7:21 am EST

Thu, October 7th, 2010Lalith J says:

My website, srilankatravelbooking.com is down and website is unreachable, Other domain colomboairporttransfer.com is also unreachable.
How long it will take to correct this problem?

Link | Posted at 7:22 am EST

Thu, October 7th, 2010Cassius Krendler says:

Yes, what is this “Chinese IP addresses being blocked”? It seems IP addresses are being blocked all over the world. In fact, not a single one of my websites seems to be working from any location I am having tests done at. Right now Pingdom.com has reported the sites being down from ALL its world wide locations.

Can we please have an ETA?

Link | Posted at 7:25 am EST

Thu, October 7th, 2010Dave G says:

IX. The event description given seems to be more than you suggest. http://www.cisco.com/en/US/tech/tk59/technologies_white_paper09186a0080174a5b.shtml
You appear to be blocking EVERY Inbound IP, or else it is more a case that there is a specific problem with particular servers in your facility. C’mon guys, update please.

Link | Posted at 7:26 am EST

Thu, October 7th, 2010Safudin says:

This is just not right. How can this be. DDos to a webhosting company server? What happen to your failover system?

Link | Posted at 7:29 am EST

Thu, October 7th, 2010Richard says:

Why are the nameservers not load balanced to protect against this?

Link | Posted at 7:31 am EST

Thu, October 7th, 2010Dave G says:

Haven’t seen any reports of other Hosting Orgs being hit.
Is the attack targeted at a wide range of IX IPs, or is it targeted at a small specific target which IX is hosting?

Link | Posted at 7:33 am EST

Thu, October 7th, 2010Saurabh says:

None of my websites is opening from any location……

Link | Posted at 7:40 am EST

Thu, October 7th, 2010黄万权 says:

快

Link | Posted at 7:44 am EST

Thu, October 7th, 2010Tim says:

do you guys test your website? I found there are at least 1-2 hours each day, my website down. What happen,

Link | Posted at 7:46 am EST

Thu, October 7th, 2010Nathan Goss says:

DDoS Attack seems like a convenient excuse to cover a bigger issue with your servers.

If it really was a DDoS Attach, then it would have been resolved HOURS ago.

Link | Posted at 7:49 am EST

Thu, October 7th, 2010IX: Brian Silhanek says:

As I understand it, we are blocking every inbound IP from one of our bandwidth providers, so it may seem like that. Customers on our other providers are able to access with no problems. We will let you know once we have more info.

-Brian

Link | Posted at 7:49 am EST

Thu, October 7th, 2010Nathan Goss says:

Are you getting info via homing pidgeons? It’s taking WAY too long!

Link | Posted at 7:50 am EST

Thu, October 7th, 2010Tim says:

http://bearsplacebandb.com
http://flyingedward.com
http://chipsfood.com/
http://coloredcode.com/
http://commonplaceblog.org/
…

Which one have my experience?? more than 10 websites on IX, all of them down. God.

Link | Posted at 7:51 am EST

Thu, October 7th, 2010Nathan Goss says:

You need to update the servers that are affected.

I have a VPS server also, and this is now down.

http://www.fulontri.com is a Shared Windows Hosting account
web.fulontri.com is on a VPS Linux box, with a dedicated IP.

Somebody appears to have royally screwed up at IX today!

Link | Posted at 7:54 am EST

Thu, October 7th, 2010IX: Brian Silhanek says:

Initially, the attack appeared to be originating exclusively from IP addresses in China. Unfortunately, as it has progressed it has moved to other locations which has led to us blocking IP addresses from one of our bandwidth providers. This bandwidth provider is the provider that most of our customers use to reach us.

We have no estimated time to resolution at this time. We will let you know as soon as we have more information

Link | Posted at 7:55 am EST

Thu, October 7th, 2010Dave G says:

Brian.
Is that the optimum way to deal with this issue. I would like to also support what Tim said in his post. We seem to have way too many issues with IX. I’d love to see the resilience and disaster recovery plan.

Link | Posted at 7:56 am EST

Thu, October 7th, 2010Leonard says:

My site is down again.

Link | Posted at 7:56 am EST

Thu, October 7th, 2010geoff says:

hey i cant get on from Australia we need it working asap its costing us $$$

Link | Posted at 7:59 am EST

Thu, October 7th, 2010Nathan Goss says:

Switch our servers across to another bandwidth provider whilst you have blocked our default bandwidth provider.

Seems a little dangerous for us to have a single point of failure with our allocated bandwidth provider.

Link | Posted at 8:01 am EST

Thu, October 7th, 2010Nathan Goss says:

One thing that I also find curious – you always manage to keep your own websites up – why can’t we be afforded the same?

Link | Posted at 8:03 am EST

Thu, October 7th, 2010Nathan Goss says:

And is it just a coincidence that the Maintenence on CP9 was immediately prior to these issues? It’s the same server, if I am not mistaken?

Link | Posted at 8:06 am EST

Thu, October 7th, 2010Leonard says:

What’s the update on this one ?

Link | Posted at 8:06 am EST

Thu, October 7th, 2010Dave G says:

IX Guys. I know you are really up to your necks dealing with this. Would you like me to file the incident report with the Department of Homeland Security US-CERT Cybersecurity bureau for you? They don’t appear to know about it yet!
https://forms.us-cert.gov/report/

Link | Posted at 8:09 am EST

Thu, October 7th, 2010Dave G says:

I have some good contacts in Juniper and Symantec. Would you like me to see if I can get you a discount on some decent security kit ?

Link | Posted at 8:11 am EST

Thu, October 7th, 2010Nathan Goss says:

I have some contacts in a very good IT recruitment firm – do you want me to get you some decent staff?

Link | Posted at 8:12 am EST

Thu, October 7th, 2010Avik says:

I’ve two sites hosted with IXwebhosting. both of the sites are not running right . Please tell me how long it would take to resolve the issue? Solve the prob A.S.A.P.

Link | Posted at 8:14 am EST

zecoo

看不懂蝌蚪文

rogerskys

看不懂。。。 用翻译软件翻译得乱七八槽

Yorgol

我已经问过了,估计还需要一段时间,现在用袋里可以访问,不过也仅仅是IP访问.

http://status.ixwebhosting.com/2010/10/07/ddos-attack-3/

官方的意思,要是能打开这个页面,说明 你的IP就不在屏蔽之内,就可以用IP进行访问.

syh863718

打不开，我被屏蔽了

rogerskys

估计还需要一段时间？  是一天，还是两天？

gypworks526

昨天下午已经好了。