Pickling and Scaling

https://pythonprogramming.net/pickling-scaling-machine-learning-tutorial/

I’ve covered Pickling before and gone through the steps – so not a lot of new stuff to cover.

Key elements being:-

Writing a Pickle

clf.fit(X_train, y_train) # fit the data to the training data
with open('LinearRegression.pickle','wb') as f:
    pickle.dump(clf, f)

Reading a Pickle

pickle_in = open ('LinearRegression.pickle', 'rb')
clf = pickle.load(pickle_in)

The other key concept on this tutorial was scaling on rented host systems. With the work flow being:-

• Rent a small part of a server to write the code (minimal cost)
• Once coding is complete – scale up the server to do the hard data crunching
• Save the output as pickles
• Scale the server back down to minimum needed

This means that you don’t need to pay for the large server all the time.

Nice idea.