If I want to build a custom database, how could I? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.

---

Want to improve this question? Update the question so it focuses on one problem only by editing this post.

Closed 4 years ago.

开发者_如何学Go Improve this question

I built a previous program that took client info and stored it in a folder of txt files (impractical much) but now I want to upgrade the program to be more efficient and put the info into a database of some sort...

How can I take the info from the text files and add them to the new database without having to manually do each one. I know this is vague but I need more so the method/logic instead of the exact code, Also if I don't use SQL what is another method for making a db (Not using another commercial Db)

btw the txt files are in simple format (name,city,age) all on separate lines for easy iteration

---

The free and portable Python shelve module in the standard library is probably all you need. It allows you to create and use what are essentially persistent dictionaries, so there's a very gradual learning curve. Converting your text files into one should be fairly easy, although won't be automatic -- you'll probably need to write a simple script to do it.

---

Well, you could read your txt* files using the csv module in Python.

I'm afraid that knowledge of SQL is a must for any sort of database manipulations, unless you have the comfort of an ORM eg. Django's ORM.

*they aren't called that on anything but Windows.

---

I would just get list of all *.txt files is directory using os.listdir() then read and parse all of them and finally put all information in some "database". Python has few such "database" kind of modules

• csv module (http://docs.python.org/library/csv.html)
• bsddb (this is only in Python 2.x, https://docs.python.org/2/library/bsddb.html)
• sqlite3 module (http://docs.python.org/library/sqlite3.html)

---

In addition to the suggestions above, take a look at Elixir, which is an abstraction layer over SQLAlchemy that makes building database interfaces (and inserting / querying information) rather simple.

---

The main reason for DB to have a SQL is to make it separate and generic from the application that you are developing.

To have your own DB built you need to have a storage mechanism could be files on the hard disk, with search options so that you can access data immediately with keywords that you are interested in. on top of this you have to have a layer that initiates queues, reads them and translates to the lower file read and write functions. you need to have this queue layer because lets say you have 100 applications and all are trying to read and write from the same file at the same time and you can imagine what can happen to the file . there will be access denied , somebody using it, data corrupted etc etc.. so you need to put all these in queue and let this queue layer translate things for you.

to start with start from different ways of reading/writing/sorting of data into the file, and a queue layer. From there you can build applications.

The queue layer here is similar to the client that is trying to push the data into the communication port in most of the available databases.