开发者

Time series database for java?

开发者 https://www.devze.com 2023-02-04 01:59 出处:网络
I want to store millions of time series entries (long time, double value) with Java. (Our monitoring system is currently storing every entry in a large mySQL table but performance is very bad.)

I want to store millions of time series entries (long time, double value) with Java. (Our monitoring system is currently storing every entry in a large mySQL table but performance is very bad.)

Are there time series databases implemented i开发者_JS百科n java out there?


  • checkout http://opentsdb.net/ as used by StumbleUpon?
  • checkout http://square.github.com/cube/ as used by square

I hope to see additional suggestions in this thread.


The performance was bad because of wrong database design. I am using mysql and the table had this layout:

+-------------+--------------------------------------+------+-----+-------------------+-----------------------------+
| Field       | Type                                 | Null | Key | Default           | Extra                       |
+-------------+--------------------------------------+------+-----+-------------------+-----------------------------+
| fk_category | smallint(6)                          | NO   | PRI | NULL              |                             |
| method      | enum('min','max','avg','sum','none') | NO   | PRI | none              |                             |
| time        | timestamp                            | NO   | PRI | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
| value       | float                                | NO   |     | NULL              |                             |
| accuracy    | tinyint(1)                           | NO   |     | 0                 |                             |
+-------------+--------------------------------------+------+-----+-------------------+-----------------------------+

My fault was an inapproriate index. After adding a multi column primary key all my queries are lightning fast:

+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| job   |          0 | PRIMARY  |            1 | fk_category | A         |          18 |     NULL | NULL   |      | BTREE      |         |               |
| job   |          0 | PRIMARY  |            2 | method      | A         |          18 |     NULL | NULL   |      | BTREE      |         |               |
| job   |          0 | PRIMARY  |            3 | time        | A         |   452509710 |     NULL | NULL   |      | BTREE      |         |               |
+-------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

Thanks for all you answers!


You can take a look at KDB. It's primarily used by financial companies to fetch market time series data.


What do you need to do with the data and when?

If you are just saving the values for later, a plain text file might do nicely, and then later upload to a database.

0

精彩评论

暂无评论...
验证码 换一张
取 消