It seems the mantra that we hear from clients most often is "...[it] must be a database problem." The reality is that it almost never is, in fact, a "database problem!" It seems that the DBA is most often "guilty until proven innocent," rather than the other way around. At first, I thought maybe this was how MySQL was viewed because of the whole open-source stigma, but I have seen the same reaction on other DBMS platforms. After chatting with Oracle, SQL Server, DB2, etc., DBA's, I heard the same thing repeated over and over...
Let me give you some of the examples I have seen. Maybe you can identify with some of these. Either way, perhaps it will generate a little smile. Naturally, this will be a short list of some of the problems and not the whole list:
Not too long ago, I was asked to troubleshoot a "problem" with the database where an ESB server could not connect to a MySQL server. I was told there was an issue with the database since they were getting connection issues! I logged into the VPN and immediately tried to login via the CLI into MySQL from localhost. No problem -- as I expected. Then, I tried to login from another machine via TCP/IP into the same database server and again no problem. Since I did not have direct access to the ESB server, I had another techie try a telnet from the same machine and they got a response! I double-checked the MySQL user table and it was set to allow connections from the ESB server! After having everything check out, I had them send me the XML config file for the ESB server to see the connect string and guess what!?!? The connect string was for a SQL Server in another country that was not even on this network!
The person who set it up copied a config file from another server and missed changing significant portions of the config! You would think the client would blush at this point and apologize, but that isn't what happened. They still insisted it was a MySQL problem! I was shocked! I had to copy the configuration section and send it to them color-coded and ask why it was trying to connect to a server in abother country that wasn't even on this network, to a database that was a SQL Server database, on a SQL Server port with MySQL configuration specifics! I think the ESB person finally saw the problem. No wonder why there was a problem! Needless to say, "database problem" solved!
There is so much bias out there that sometimes others are blinded to the obvious in their haste to blame someone else! I presume the same cry can be heard from Development Teams, Unix administrators, and so on...
I have also seen a number of networking problems mis-diagnosed as database issues. Perhaps you have been there too. I have seen ailing network cards that were becoming intermittent cause issues (not to mention switches with bad ports). While these hit-and-miss errors are hard to diagnose, they are rarely MySQL problems. But, as always, the other tech people responsible for those platforms have learned the mantra -- "got to be a database problem..."
Another common issue is when an application person decides to copy the JBoss / Tomcat configuration over from a Dev box to a new UAT server and forgets that we keep user access permissions different on the various environments for security reasons. As you can imagine, when they start up and get a connection error, it is time to call the DBA and figure out what is wrong with the database! I have learned to make my first request be for seeing their configuration file!
Anyway, I could rant on and on, but I was curious whether anyone else noticed the same pattern? Surely, it isn't just me'! Am I the only DBA that gets called in every so often in the middle of the night to diagnose someone else's problem? I sure hope not! Maybe "misery" really does love company...
Tracked: May 15, 10:29