开发者

Json vs. SQL Recommendations

开发者 https://www.devze.com 2023-03-03 14:45 出处:网络
I don\'t have that much programming experience beyond python and I am starting a project that is going to store info by a city/country, date, and then time.I have used simplejson before and find it in

I don't have that much programming experience beyond python and I am starting a project that is going to store info by a city/country, date, and then time. I have used simplejson before and find it incredibly useful, especially because how easy it is to sort through the json data with python.

If I used json I would have a dictionary with 30 different keys (for each city/country) and then for each country or city I would have the value be a new dictionary with the date as a key and the value of that would be another dictionary with the time as keys. The value of each time key would be a list of 10 different dictionaries (I am sorry if I didn't not say that more concisely). I plan to keep this data for several years and the data is mainly just text. I don't know how much text there will be, but there will be A LOT of text. The data will be updated several times an hour. I will be the only accessing the data and I may eventually put the data online for people to view.

So the data structure might look something like this:

data = {
  'Toronto, Canada': {
    '2011-05-04': {
      '00:30': [
        { ... },
        { ... },
        .开发者_运维知识库..
      ],
      '02:00': [
        ...
      ],
      ...
    },
    '2011-05-05': {
    },
    ...
  },
  'Tokyo, Japan': {
    ...
  },
  ...
}

I am starting to teach myself SQL but what would anyone recommended? Should I create the json structure that I described above (or some altered version, I am up for any suggestions) or should I create a database through SQL?

Also I will be analyzing the data with different python functions so I don't know if that impacts any suggestions.


JSON is for exchanging smallish amounts of data between processes on the same machine or over the web. You need a database.

If you use JSON, you need to read the whole structure into memory before you can query it or update it. You need to write the whole lot back to disk after each update (or risk losing data when the power fails). You will find that analysing your data will be much easier using SQL than with JSON.

A few suggestions:

(1) Use SQLite (comes with Python)

(2) Consider having city and country as separarate columns. Consider adding a "state/province" column.

(3) Consider whether date and time should be one "timestamp" column.

(4) You haven't given any detail on the actual data other than saying it is a "list of 10 dictionaries". Things you need to be sure of (and might like to ask for help with) are: Is that list really a set? If it really is a list, what is the meaning of the list subscript? Are all of the possible dict keys known in advance (and how many are there), or are they variable data? The answers will effect how you design your table(s).

0

精彩评论

暂无评论...
验证码 换一张
取 消