Chase not being flexible with customers hurt by the outage

On Wednesday, a Chase official made the following statement:

“If a customer incurred a late fee during this outage, they should call us via telephone banking or go into a branch, and we will rescind that late fee,” she said. “If they incurred these fees during the outage because they could not make a payment any other way, then it is our responsibility and we will refund that late fee.”

The language sounds like it is specifically crafted to give them an easy out by just claiming a customer didn’t try every other way to make a payment, and it looks like they are already availing themselves of this.  Here is a comment we received:

I finally was able to get in yesterday afternoon as I needed to pay my 2 cards – they were due on the 15th.  It was after the 5 p.m. EST cut off time (I’m on PST).  I called and got customer service, as I wanted to know if I could pay at a branch. My only option: pay $14.95 to have them take the payment and process effective the due date. $14.95 per credit card. I wanted to know how they could charge this fee when they have been down. Her response: you had the option to call our number and pay via our automated system before 5 p.m. EST.  Great.  I always pay online from where I’m at. I didn’t have any of my info (phone #, credit card #, bank account #).  Seriously sucks.

C’mon Chase, you seriously messed up and not everyone had time to wait on an extended hold on you seriously backed-up telephone banking or make it into a branch.  You should be waiving fees like this left and right.

Chase online failure – after effects

The comments and Twitter are quiet about new problems so I think we can assume that the full (Monday, Tuesday) and partial (Wednesday) outages of Chase’s online banking are over for now.

The Wall Street Journal has a good article on the problems worth reading.

Chase’s biggest failure throughout this mess, and in its aftermath, is the failure to inform its customers what is going on.  First of all, they tried to pass it off as scheduled maintenance, in the middle of the day.  Then they didn’t officially acknowledge the problem, other than the notice on their site.  When problems reappeared on Wednesday, they were very slow to acknowledge them.

During the outage the very limited information they released did not indicate anything about the nature of the problem, how people might be ultimately affected, and how long the problem might last.  They failed to use the best possible channel to get any information to their customers, their website, to release information.

Lastly, since the problem was fixed, they have not come out with a clear explanation of the cause of the problem and what, if any, customer data or worse, funds, might be at risk, or have indicated that such a statement will be forthcoming.  The only information they provided to customers after the fact was a short apology and an incomplete description of what would happen to one type of transaction scheduled during the outage.

In short, this outage has been a complete communication failure for Chase as they left their customers completely in the dark.

We are getting a limited number of reports from people who are finding fraudulent activity in their accounts.  It is unclear at this time whether this might be a more widespread problem, whether this is normal fraud that is incorrectly being associated with the outage, or whether what appears to be fraudulent activity was caused by data corruption.

A short while ago they updated the message available from their main page from the previous short apology to a longer message similar to that which was available to customers once they logged in.  Here is the message:

We are sorry for the difficulties that recently affected Chase.com, and we apologize for not communicating better with you during this issue. Giving you 24-hour access to your banking is of the utmost importance to us. This was not the level of service we know you expect, and we will work hard to serve you better in the future and to communicate with you better if a situation like this should arise again.

Online Bill Payments scheduled for September 13, 14 or 15 were processed by Wednesday night, September 15. It is not necessary to reschedule these payments. If you scheduled a payment during those dates, but do not see it reflected in your payment activity by September 16, please contact us.

We will refund any late fees that you may have incurred as a result of our delay in processing your payment.

Thank you for your patience and for the opportunity to work harder to serve you in the future.

They are now additionally apologizing for the lack of communication and claiming to do better in the future.

This site (chase-sucks.org) was created in the hopes of helping convince Chase that they needed to do better at serving customers (more service, less shenanigans).  I can only hope that the black-eye this outage, and their response to it, has given Chase will truly convince them to do better for their customers.

Update Thursday 9/16/10:  Chase has not change the top banner on their main page to duplicate the apology link, as if saying it twice makes it carry more weight.  The apology text itself has not changed.

Update Saturday 9/18/10:  Here is a report from a customer saying their debit card stopped working as a results of the Chase outage:

I tried to use my debit card for an online transaction today and it was declined.  This immediately worried me given the problems they’ve been having, so I tried to withdraw money at an ATM.  An error message came up at the ATM saying that my card information was “incomplete.”  Whatever happened to their system has clearly scrambled their databases.  I live overseas and rely on my ATM card to access my money.  No working debit card=no way for me to get money, except for an expensive credit card advance.  The best part?  They are saying that they need four or five days to even get back to me on this.

Chase back up for good?

Sometime around 9 or 10 pm ET or so Chase online banking seems have stabilized, which we are estimating by when we got our first report in the comments and when the “Chase is down” tweets stopped.  Chase hasn’t provided an official time when the problem was fixed.

They have posted a short apology on their website but it contains no info on the cause of the problem:

To our customers:

Please accept our apology for the difficulties that recently affected Chase.com.

Giving you 24-hour access to your banking is of the utmost importance to us. This was not the level of service we know you expect.

Thank you for your patience and for the opportunity to work harder to serve you in the future.

Their spokesman released a brief statement about the nature of the problem:

“A third-party database company’s software caused a corruption of systems information, disabling our ability to process customer log-ins to chase.com. This resulted in a long recovery process,” Chase spokesman Thomas Kelly said Wednesday evening.

Clearly log ins were affected.  It remains to be seen whether reports of other fallout problems will surface when people start using the service in earnest on Thursday.  A more detailed statement was apparently available to customers once they logged in:

Please accept our apology for the difficulties that recently affected Chase.com.

Online Bill Payments scheduled for September 13, 14 or 15 have been processed or will be processed Wednesday night. It is not necessary to reschedule these payments. If you scheduled a payment during those dates, but do not see it reflected in your payment activity by September 16 please, contact us.

We will refund any late fees that you may have incurred as a result of our delay in processing your payment.

Giving you 24-hour access to your banking is of the utmost importance to us. This was not the level of service we know you expect.

Thank you for your patience and for the opportunity to work harder to serve you in the future.

It is also highly likely that the Chase’s online banking system will be getting higher than usual volume Thursday as people log in to check their accounts.  It will be interesting to see what happens.

Chase outage now at 48 hours

For Chase online banking customers the only news is the lack of news and specifically no new official statements (or any significant statements) from Chase as to the nature, severity, effects, and potential duration of the problem. For a blow-by-blow of today’s events, see our earlier post.

In the absence of any official statements by Chase, we have had to rely on unsubstantiated reports that we’ve seen in our comments and through Twitter.  One such tweet recently suggested a fairly severe problem:

Chase outage is a result of corruption to the database clusters containing customer online profiles (ie bil pay settings, usernames etc.).

Depending on the reliability of Chase’s backup system, this could result in some pretty inconvenient loss of data for customers.

The outage, which by our best estimates started sometime between 7-10 pm ET on Monday, and has included online banking being completely disabled/unavailable from Monday evening until early Wednesday morning, and sporadic and intermittent access since Wednesday morning, has now exceeded 48 hours and is believed by many to be the longest duration online banking outage for a bank of Chase’s size.

Chase online banking still experiencing problems

We’ve received quite a few comments (starting at around 6:40 am PT/9:40 am ET) that Chase’s website is still experiencing problems.  The most common experience seems to be the site timing out after someone tries to log in.  Some people were able to log in earlier but are having problems now.  One reader reports his login being rejected despite entering the correct user name and password.

One customer received the following error, clearly from the servers behind the scenes:

This error code is native to internal exceptions from the Oracle programs. It indicates that the process has encountered an unexpected low. It may be due to
•
•timeout
•A corrupted file
•crash verification data in memory
•problem of I/O of physical memory

Perhaps this is why they haven’t offered an official explanation … they haven’t really solved it yet.

Update 8:20 am PT/11:20 am ET:  Its officially down again.

As a side note, when I was looking at the website source for the main page this morning while the site was still up, I noticed that the language for the site being down was still in there, just not active.  This might be a sign that Chase has known since they put the site up again last night that they didn’t fix the problem.

Update 9:00 am PT/12 noon ET:  The login screen is active again.  I think it is safe to assume you can’t count on the site for the time being.

At this point (and well before this point) Chase has completely failed in its responsibility to get information out to its customers.

Update 9:30 am PT/12:30 pm ET:  Still receiving reports of log in failures.  When I try to load the main page (www.chase.com) it never quite finishes loading

Update 11:30 PT/2:30 ET:  Still receiving reports of inability to log in.  Does Chase think that having the login screen present despite widespread inability to log in is less of a black eye than having an system down message on their site?  The media is reporting Chase online banking as problematic regardless.

Update 1:30 pm PT/4:30 pm ET:  Still receiving reports of inability to log in.  Not fixed yet.

Update 3:30 pm PT/6:30 pm ET:

• Problems persist.
• Chase is acknowledging that there are still problems (gee thanks).
• I’ve seen several reports that this is now the longest outage ever of a bank this size.
• Chase has no comment on why it is not communicating with customers what is going on.

Update 4:30 pm PT/7:30 pm ET:

Still lots of reports of the same problems.  No new information of any kind from Chase.  Their lack of public statements about this problem is disappointing, and this far into it very irresponsible.

I’ve seen several comments from people claiming that they are seeing unauthorized activity on their accounts, in the brief moments when they can actually see their balances.  Another reader contacted me with this story:

I was able to log into my account this morning, but that balance was all wrong. I had double charges, and the calcuations didn’t add up right. I have NO IDEA how much I have in the bank right now. I’m glad I opened a new account a few days ago at a different bank. I’m just waiting for Chase to get their shit together so I can close the account. I feel like a hostage.

I don’t think there is enough information to make a significant generalization from these comments, but the potential trend is worrisome enough to mention.

Update 5:30 pm PT/8:30 pm ET:  Just saw this tweet:

Chase outage is a result of corruption to the database clusters containing customer online profiles (ie bil pay settings, usernames etc.).

This is completely hearsay, but without Chase providing any official statements to clue people in, this is what we have to go with.  It is surely a coincidence that I found quite a few listings for open database administrator jobs JP Morgan Chase has posted.

Chase online banking outage – 10 pm PT/1 am ET update – IT’S UP!

Our test server logs show that at 10:03 pm PT/1:03 am ET the chase.com server abruptly closed a connection to our test program and this is presumably when they brought the site back up.  It is now 10:30 pm PT/1:30 am ET and the chase.com main page is once again showing the login fields.  Ok so they squeaked by with their statement that the service would be back up “this evening”.

That puts this outage at around 28-30 hours long.

Since I don’t personally have a Chase account, I can’t try to log in and check the online banking and credit card services.  If you are a Chase customer and have any kind of experience using the website now, please leave word of what you find in the comments.

Chase online banking down – 7:45 pm PT/10:45 pm ET update

According to this article in the Wall Street Journal, Chase is just now clarifying the nature of the problem and the time frame for online banking to come back up:

A spokesman for the nation’s second-largest bank by assets said the service went down because of a “technical issue, not hackers or criminals.” Another person familiar with the situation said the disruption was the result of a flaw in a software program tailored for J.P. Morgan.

It took the bank much of Tuesday to discover the flaw and figure out how to fix it across the system, this person said. The bank expected the online service, which affected computer and iPhone users, to be fully restored Tuesday evening, this person added.

It is now almost 11 pm on the east coast, so they are straining the definition of evening, but perhaps they will claim west coast or Hawaii time to justify this statement.  We’ll see if it comes back up before Tuesday rolls to Wednesday here on the west coast.

They still have not provided any more detailed information on their website, the most obvious and efficient place to distribute up to date information on this problem.

Chase online down – 4 pm PT/7 pm ET update

More comments from Chase’s spokesman, Tom Kelly:

The bank will work with any customers who had online bill payment deadlines they couldn’t meet because of the outage, Kelly said.

We’ve had one customer report in the comments that an incoming ACH transfer was successful, but according to Kelly’s statements, we can assume there are problems with online bill payments.  It might be a good idea to contact your payees or make other arrangements (like a good old fashioned check in the mail).

Again in summary from our 1 pm PT/4 pm ET update, branches, ATM’s and telephone banking are operational although telephone banking still has a considerable wait.

In another statement Tom Kelly says:

The cause of the outage isn’t clear, Kelly said. “It’s a technical issue that we’re dealing with.”

So, clearly they won’t be able to give any estimate on when the online banking might be back up.

If anyone is interested, this is the contact information I found for Chase’s spokesman:

Thomas Kelly
Chase Media Contact
(312) 732 – 7007
cell: (312) 848-5265
thomas.a.kelly@chase.com