MySQL takes a long time (hours) to determine a lost connection_问答_开发者

We have a service that hits a few dbs on the sam开发者_JAVA百科e separate server. The service went down, but mysql didn't abort all of the connections, and kept one open for a very long time. Below is a mysql log. We can see based on the connection id that the connection reported on the last line should have been aborted with the rest of them at 00:01, but it hung around for over 2 hours.

110216  0:00:01 [Warning] Aborted connection 11864307 to db: 'foodb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11865442 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11864559 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11863856 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11865445 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11864581 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11865447 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11864566 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11865451 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11865446 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11865444 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11864565 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11865490 to db: 'foodb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11864251 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11864299 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11864303 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11865486 to db: 'foodb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11864304 to db: 'foodb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11864560 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11864268 to db: 'foodb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11865055 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11865443 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:01 [Warning] Aborted connection 11865450 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:02 [Warning] Aborted connection 11865448 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  0:00:02 [Warning] Aborted connection 11865485 to db: 'foodb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)
110216  2:11:17 [Warning] Aborted connection 11865449 to db: 'bardb' user: 'db_user' host: '10.234.1.15' (Got an error reading communication packets)

Now, assuming this is a problem with mysql, given the below timeout settings, is there anything that could possibly be causing this?

mysql> show variables like '%time%';
+----------------------------+-------------------+
| Variable_name              | Value             |
+----------------------------+-------------------+
| connect_timeout            | 10                |
| datetime_format            | %Y-%m-%d %H:%i:%s |
| delayed_insert_timeout     | 300               |
| flush_time                 | 0                 |
| innodb_lock_wait_timeout   | 600               |
| innodb_rollback_on_timeout | OFF               |
| interactive_timeout        | 28800             |
| lc_time_names              | en_US             |
| long_query_time            | 5                 |
| net_read_timeout           | 30                |
| net_write_timeout          | 60                |
| slave_net_timeout          | 3600              |
| slow_launch_time           | 2                 |
| system_time_zone           | GMT               |
| table_lock_wait_timeout    | 50                |
| time_format                | %H:%i:%s          |
| time_zone                  | SYSTEM            |
| timed_mutexes              | OFF               |
| wait_timeout               | 28800             |
+----------------------------+-------------------+

mysql> show variables like '%packet%';
+--------------------+----------+
| Variable_name      | Value    |
+--------------------+----------+
| max_allowed_packet | 16777216 |
+--------------------+----------+

Your max_allowed_size must be default 1M and did you made some query with huge data, blob or something. Increase that value to 8M (1GB is max limit).

Reduce wait_timeout to 50 or so.

wait_timeout is "The number of seconds the server waits for activity on a noninteractive connection before closing it. This timeout applies only to TCP/IP and Unix socket file connections, not to connections made using named pipes, or shared memory." Details