Ok, so we move forward, a little bit anyway. Since we don't have any extra cash sitting around we needed to make use of some free stuff and other stuff sitting around.
A database server was needed to house the 100 million plus records as well as perform the large array of statistical queries we plan on running. Linux has definitely come a long way. We chose an olrder Ubuntu Distro since I already had a CD burned. We used an older 2GHZ AMD Athlon box with 512MB of RAM, one that usually serves as my test box for various projects.
So for $0 and about 2 hours we had a nice little MySQL database server ready to accept a ton of data! Tonight I wrote a simple program in c# to convert and load all the test data into our new database server. So far we are getting about 1 Million records loaded per hour so this will take a few days to complete. In the meantime we'll start working on some recipes for our predictor algorithm and start testing it on the live data as it is loaded. I probably should have transformed the data then direct loaded into MySQL instead of from my XP client via ODBC but oh well, the thing about this whole project is for it to be fun!
Onward we go...
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment