开发者

Best way to store large data in mysql

开发者 https://www.devze.com 2023-03-18 14:59 出处:网络
I need to 开发者_开发技巧store large amount of data every hour in the database. What kind of data? Text data.

I need to 开发者_开发技巧store large amount of data every hour in the database. What kind of data? Text data.

What is the best way? Store on multiple table or 1 large table?

Edit: I just said, large text data. 10000 times the word "data"

Every hour a new line is added like: hour - data

Edit 2: Just because you can't understood the question, and also i said, "EVERY HOUR", so you imagine every hour for the next 10 years a new line will be created, does not mean its not a readable question.


Use a column of datatype 'text', 'mediumtext', or 'largetext' according to your needs.

See: http://dev.mysql.com/doc/refman/5.0/en/blob.html

Alternatively, you could just output the data to a file. They are more appropriate for logging large amounts data that may not need to be accessed often - which it seems like this might be.


MySql have added many feature in MySql 5.7. Now you can do it in many way.

Oracle like Big Data is now Integrating in MySQL.

MySql have Unlocked New Big Data Insights with MySQL & Hadoop.

Soluation 1: You can use MySQL as a Document Store. There are possible to store many many object as JSON. It highly recommended and Extendable.

MySQL Document Store = (MySql + NoSql).

X Dev API will help to produce JSON with SQL and CRUD operation over X Protocol. Also there is possible to maintain X Session.

It will be best for transparent data sanding and sharing for chat application or group Application.

Soluation 2: MySql Sysbench: Read Only is another best solution. It will be very very fast and scalable to make group chat Application.

Soluation 3: Use MySql 5.7 : InnoDB, NoSql with Memcached API which will interact directly with storage engine InnoDB. It is 6X faster than MySql 5.6.

Still Now FaceBook is using this technology. Because it is very fast.

For more details:

https://www.mysql.com/news-and-events/web-seminars/introducing-mysql-document-store/
https://dev.mysql.com/doc/refman/5.7/en/document-store-setting-up.html
About Big Data: https://www.oracle.com/big-data/index.html
https://www.youtube.com/watch?v=1Dk517M-_7o


I think it is better to use a database that is not used by anything else but whatever uses the data (as it is a lot of text data and may slow down SQL queries) and create seperate tables for each category of data. Ad@m

0

精彩评论

暂无评论...
验证码 换一张
取 消

关注公众号