Machine Learning – Tutorial 26

Support Vector Machine Optimization in Python


More resources:-


import matplotlib.pyplot as plt
from matplotlib import style
import numpy as np

# build SVM class

class Support_Vector_Machine:
    # The __init__ method of a class is one that runs whenever an object is created with the class
    # calling self in the class allows sharing of variables across the class, so is included in all function defs
    def __init__(self, visualisation=True):
        # sets visualisations to what ever the user specifies (defaults to True)
        self.visualisation = visualisation
        # defines colours for the two states 1 & -1
        self.colors = {1:'r', -1:'b'}
        # sets some standards for the graphs
        if self.visualisation:
            self.fig = plt.figure()
            self.ax = self.fig.add_subplot(1,1,1)
    # train
    def fit(self, data):
        # set up access to the data that's passed when the function is called
        self.data = data
         # { ||w||: [w,b] }
        opt_dict = {}
        transforms = [[1,1],
        # finding values to work with for our ranges.
        all_data = [] # set up a placeholder for the values
        # for loop to step through data and append it to all_data (list of values)
        for yi in self.data:
            for featureset in self.data[yi]:
                for feature in featureset:
        # next define the max and min value in list
        self.max_feature_value = max(all_data)
        self.min_feature_value = min(all_data)
        # free up memory once we've got the values
        # define step size for optimisation Big through to small
        step_sizes = [self.max_feature_value * 0.1,
                      self.max_feature_value * 0.01,
                      # starts getting very high cost after this.
                      self.max_feature_value * 0.001]      

        # extremely expensive
        b_range_multiple = 5
        b_multiple = 5
        # first element in vector w
        latest_optimum = self.max_feature_value*10

        ## Begin the stepping process
        for step in step_sizes:
            w = np.array([latest_optimum,latest_optimum])
            # we can do this because convex
            optimized = False
            while not optimized:
    def predict(self,features):
        # sign( x.w+b )
        classification = np.sign(np.dot(np.array(features),self.w)+self.b)

        return classification

# define data dictionary
data_dict = {-1:np.array([[1,7],


Machine Learning – Tutorial 14

Applying K Nearest Neighbors to Data


Data links:-


# import libs
import numpy as np
from sklearn import preprocessing, neighbors
# cross_validation is depreciated and train_test_split moved into model_selection
from sklearn.model_selection import train_test_split
import pandas as pd

df = pd.read_csv('breast-cancer-wisconsin.data')
# there are gaps in the data denoted by '?' - these need to be converted to -99999 so the algorythm treats it as an outlier
# drop any useless data - in this case the ID

# define X & y (X for features; y for labels)
# X is everything except 'class'
# In the datafile I had a space after 'class' which caused errors
X = np.array(df.drop(['class'], 1))
y = np.array(df['class'])

# split the data into train and test datasets using train_Test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)
#define classifier (clf)
clf = neighbors.KNeighborsClassifier()
# fit the classifier
clf.fit(X_train, y_train)

accuracy = clf.score(X_test, y_test)


# important the array needs to be 2D so double brackets are needed rather than reshaping the array
example_measures = np.array([[4,2,1,1,1,2,3,2,1],[4,2,1,2,2,2,3,2,1]])
prediction = clf.predict(example_measures)

Machine Learning – Tutorial 12

Creating Sample Data for Testing


Generating and using sample data to test the functions and equations – similar to unit testing.

Key Points:-

  • When starting to define a function build out the inputs and then focus on what it needs to return – and then connect the two with the various steps
  • Once the test data is calculated we can then change the variance and correlation in the equation and test that the R^2 value changes as expected
    • create_dataset(40,10,2,correlation=’pos’) – gives a higher R^2
    • create_dataset(40,80,2,correlation=’pos’) – gives a lower R^2
    • create_dataset(40,10,2,correlation=’neg’) – draws the line the other way around
    • create_dataset(40,10,2,correlation=False) – gives a very low R^2 (no correlation)
# Import Libs
from statistics import mean
import numpy as np
import matplotlib.pyplot as plt
import random #psudo random :)

# To set charts to save as images we need to change the default behaviour
from matplotlib import style # inport style to change default behaviour of plot
style.use('ggplot') # use ggplot

# Define values
#xs = np.array([1,2,3,4,5], dtype=np.float64) # dtype lets you set the data type. Not needed for this example but useful in future
#ys = np.array([5,4,6,5,6], dtype=np.float64)

def create_dataset(how_many, variance, step=2, correlation=False): # step and correlation have default values set that can be overwritten when the function is called
    #set up starting values
    ys=[] #empty array
    # for loop cycling through how_many to give ys
    for i in range(how_many):
        y = val + random.randrange(-variance, variance)
        # also need to increase/decrease value of val depending on correlation
        if correlation and correlation =='pos':
            val += step # add the value of step to Val and store as val
        elif correlation and correlation =='neg':
            val -= step # minus the value of step from Val and store as val
    # now define xs
    xs = [i for i in range(len(ys))] # one line for loop to generate an array of ittrative xs; could have used How_Many

    return np.array(xs, dtype=np.float64), np.array(ys, dtype=np.float64)

# Define best fit function
def best_fit_slope_and_intercept(xs, ys): # defining function to calculate slope (m) - passing values of xs and ys
    m = ( ((mean(xs)*mean(ys)) - mean(xs * ys)) / # bracket space at the start and space slash at the end allows for a carridge return in the code
        ((mean(xs)**2)-mean(xs**2)))  ## **2 raises to the power of 2
    b = mean(ys) - m*mean(xs)
    return m, b

# define new xs and ys
xs, ys = create_dataset(40,40,2,'pos')

m, b = best_fit_slope_and_intercept(xs,ys)
# Define function to square error
def squared_error(ys_orig,ys_line):
    return sum((ys_line - ys_orig) * (ys_line - ys_orig)) # return used with calc rather than seperately first

def coefficient_of_determination(ys_orig,ys_line):
    y_mean_line = [mean(ys_orig) for y in ys_orig] # one line for loop
    squared_error_regr = squared_error(ys_orig, ys_line)
    squared_error_y_mean = squared_error(ys_orig, y_mean_line)
    return 1 - (squared_error_regr/squared_error_y_mean)

m, b = best_fit_slope_and_intercept(xs,ys)
regression_line = [(m*x)+b for x in xs]

r_squared = coefficient_of_determination(ys,regression_line)

plt.scatter(xs,ys,color='#003F72', label = 'data')
plt.plot(xs, regression_line, label = 'regression line')
plt.savefig('ML_Tutorial12.png', bbox_inches='tight') #Sets the output to save an image
plt.show() # exports the image

Python Basic Cheat Sheet


  • pip –version
  • python -m pip install –upgrade pip setuptools wheel
  • pip install “SomeProject”
  • pip install “SomeProject==1.4” [to install a specific version]
  • pip install –upgrade SomeProject

Covid Isolation Week 8 Day 2 – Three rules

So today has been one of those days where Isaac hasn’t really wanted to do much.

Although he did blitz through the maths assignments for this week (it was only multiplying by 10 and 100) when it came to practising his tables he wanted to do the bare minimum. Now in fairness to him, his teacher had only set them to practice 3s and 4s this week – but Isaac is confident on his tables up to 11s…so no real challenge.

It came to a head on our daily walk where, in his usual way, he was trying to get out of doing something by making stuff up. Today’s avoidance strategy was basically saying that he didn’t want to learn more than the teacher had set as it would be unfair on his friends if he learnt more than them. Reasoning and cajoling didn’t work so we got into a long rambling discussion on why it’s important to push yourself, everyday…

Which distilled down to 3 lessons:-

  1. Never give up on something (perseverance)
  2. Always aim to be better than you were the day before
  3. Don’t compare yourself to others

I kind of think I might turn these into separate posts with more detail and other bits….but for now that’s what we covered…

Covid-19 Symptom Check

We’re still persevering through it all 

Covid Isolation Week 7 Day 4 – Bad parenting

So today was a lesson in parenting traps and how it’s so easy to fall headlong into one.

Isaac is obsessed with superhero’s – unsurprisingly given he’s 7 and they’re everywhere…..everywhere…

So for a long time he’s watching youtube clips about Avengers, playing Avenger lego games, watching lego shorts based on the avengers. And today had been particularly slow (as we both had meetings and calls) so at the end of the day I’m sat down with him and he asks if he can watch Avengers.

Now he had seen Captain America while in Saundersfoot with his Thad-cu – and I thought what would the harm be if I’m watching it with him.

So I agree – he trots off, gets the film, excitedly works out how to use the XBox to play Blu-rays….and we settle in for the film.

All goes well…the Avengers meet, bicker, fight, assemble, bicker…and then finally capture Loki…

Lots of bantery exposition followed by more bickering and then it all goes to hell (we’ve all seen it)…Big Green goes crazy, explosions, crisis, engines blow up and Loki escapes…

And then Coulson pulls a gun on Loki…. and suddenly Loki is behind him delivering a seemingly fatal stab…

Isaac goes quiet…

The film continues….Thor is ejected…

Isaac is still quiet….then pipes up “There’s too much blood, I want to play minecraft!”

I try and chat to him about….but I’m confronted with a few tears and him saying he didn’t realise this was a horror.

….and at the moment I was completely floored. Yes he’s a 7 year old who’s favourite minecraft activity is slaughtering everything in sight, who loves play fighting and nerf guns. But in all of that I’d missed that he was just a 7 year old boy not prepared for the pivotal shock moment of the film…not ready to see a good guy fail and die.

Bad Dad moment…

We talked about it afterwards and he knows its all pretend….but still…lesson learnt… keep to the ratings…

Where is Coulson in the current MCU? - Quora

Covid-19 Symptom Check

Better the Coulson


Covid Isolation Week 1 Day 4 – Quiet times

It’s amazing how quiet things are getting.

During the day there are still cars around and people going for walks but at night it gets eerily quiet – no one out driving to those social events or running errands. 

One thing I’ve noticed today is far more sirens – generally far off. Maybe it’s just because there’s less traffic generally so less of the normal city sounds but 3-4 times today I noticed sirens of ambulances or police cars… I pray who ever is in need to the sirens is ok – which again might be a factor in me noticing more.  

So the new routine continues…the new norm…
Get-up, Tea, sometimes breakfast, log on, work, calls, work, more calls, make tea, lunch, catch-up, hand over, shower (I know late), walk, maths, reading, science, snack, log back on, more calls, emails, apologies, tea, work-out, bed time, dinner, TV, bed…rinse repeat.

Its harder on Isaac I think. The initial joy at not having to go to school is slowly giving way to boredom, which isn’t helped by him being an only child. But we’re trying to keep him occupied and the school have been great at setting him things to do – he’s even started a diary.

It also struck me today how I’m slowly losing track of what day it is…without those typical, familiar queues it all just becomes another day… We’re not even through week 1 and I’m sure life will get more and more surreal…

In other events, I saw my mum today when she dropped off some cardboard for Isaac to build a castle from. Oh and some random flapjacks that she made to free up space in her kitchen. Think we’ll tackle the model building on the weekend (the flapjacks didn’t last a day). I’m trying to get him to build a model of Caldicot Castle but he has his heart set on Castle Coch – because red is his fave colour. 

Covid-19 Symptom Check

Apart from the tail end of Isaac’s cold we’re all ok 🙂


NINo Stars

asterixSo the challenge today was how to make it easier for customers to get their initial password correct.
This would be based on their National Insurance number – but in lowercase, with duplicate consecutive characters replaced by an asterisk…

My idea to make this easier was to simply put some online page containing advice on how to log on initially, with a tool for easily applying the rules to a customers national insurance number.

Apparently this is complicated – needs a developer and has to be impact assessed….

Or I could just do it myself – or so I thought.

Anyway after a few minutes jogging my memory on Javascript I came up with this.

<!DOCTYPE html>

<p>Click the button to convert the string to lowercase letters.</p>
<textarea id="NINo" cols="50" name="p" rows="4">National Insurance Number</textarea>
<button onclick="myFunction()">Generate</button>

<p id="demo"></p>

function myFunction() {
   var str = document.getElementById("NINo").value;
   var res = str.toLocaleLowerCase();
   var res = res.replace(/11/g,"1*");
   var res = res.replace(/22/g,"2*");
   var res = res.replace(/33/g,"3*");
   var res = res.replace(/44/g,"4*");
   var res = res.replace(/55/g,"5*");
   var res = res.replace(/66/g,"6*");
   var res = res.replace(/77/g,"7*");
   var res = res.replace(/88/g,"8*");
   var res = res.replace(/99/g,"9*");
   var res = res.replace(/00/g,"0*");

   document.getElementById("demo").innerHTML = res;



Clunky, and I’m sure there’s a Regex version that would be tidier – but that’s for another time.



I seem to spend an awful lot of time thinking about doing things rather than just getting them done.

Procrastination… for me of late has been an art form – in all areas of my life. I’d like to believe it’s just ‘of late’ but then that would be a self indulgent lie. So time to sort it out and get back to being productive.

Even as I’m writing this there are other things that keep distracting me – no more!

So I need to get back to:-

  • reading – working the grey matter
  • exercise – working the wobbly matter
  • art – or more rightly blogging/cartoons
  • work – got to pay the bills some how

This week I’m going to clear the decks of some things that have been hanging around and start with getting back into the exercise habit.





Is this how to write spam comments?

I think someones spam-bot went a little awry and posted their script instead of just a selection…


{I have|I’ve} been {surfing|browsing} online more than {three|3|2|4} hours
today, yet I never found any interesting article like yours.
{It’s|It is} pretty worth enough for me. {In my opinion|Personally|In my view}, if all
{webmasters|site owners|website owners|web owners} and bloggers made good content as
you did, the {internet|net|web} will be {much more|a lot more} useful than ever
I {couldn’t|could not} {resist|refrain from} commenting.
{Very well|Perfectly|Well|Exceptionally well} written!|
{I will|I’ll} {right away|immediately} {take hold of|grab|clutch|grasp|seize|snatch} your {rss|rss

Continue reading %s

Contact forms… which is best

What I’ve learnt from getting back into blogging has been how very different Word Press and Blogger are as platforms. Probably for all the right reasons – it just feels a lot more grown-up and serious.

Anyway I digress (marginally) from the point of this post – which is there are so so many options around design/widgets/forms/formats/categories/tags…almost too much!
It’s similar to the reason I sometimes avoid Subway – a bewildering range of choice.

‘You can have everything!’

…yeah but what if I don’t know what I want?

So I got to reading Jeff Bullas’ blog and a post about the 15 most common mistakes…and the section on contact forms caught my attention. This site is very much a work in progress but I hadn’t thought of really using a contact form – but actually after reading Jeff’s take on it I thought I’d look at putting a simple form together.

Which brings me to choice – there are hundreds of options for plugins that handle forms from a simple html form to cloud based filtered services that will integrate with your CRM. Where’s good old safe and easy blogger when you need it.

This led me into the ‘review minefield’ – multiply the number of plugin options by the number of reviews and suddenly a simple choice is somewhere in the middle of an ocean of information. Which at the speed I read devoured a good few hours.

After looking at all those reviews and ratings what did I decide to do?

Well I listened to Jeff’s advice and installed one of his recommendations.

Moral of the story?
Sometimes choice is good, flexibility and scale-ability is always nice – but sometimes a good recommendation is far better than hours lost in research on a new subject.



One voice

For the last few months I’ve been involved in redrafting our communication strategy and yesterday was the final presentation of our recommendations. Thankfully it went far better than expected.

I’ll probably do a more in depth post later with the details but the headlines revolved around:-

  • Getting the overarching narrative of the organisation right
  • Getting people to talk to one another
  • Listen to customers and talk their language

Not rocket science but the more I have worked alongside communication teams the more I’ve realised what a rough deal they often get.

I’ve met very few people in the business world who do not rate themselves as excellent communicators.
It’s littered through most CVs I’ve ever read and is certainly on all of mine.

“Excellent communicator….proven track record of communicating at a senior level….experienced public speaker….yada yada yada”

So inherent in most people is a instinctive under appreciation for anyone involved in comms and the value they add. This often means that the comms team gets involved very late on or have pre-written copy passed to them to be sent out… and then people get offended when its recommended that half of what they’ve written isn’t needed.

So how do you sell egg sucking classes to your grandma?

Well the first tactic was to build up a comprehensive evidence base. This involved surveys, focus groups, analysis of customer data as well as bringing in some experts to give their opinion. The second tactic was to really not refer to it as communication at all…
Communication should be a two way dialogue but (certainly in my organisation) has become a shorthand for just sending an email on whats going on – hardly two way. So when the ‘name’ gets in the way…change it…

We presented a ‘One Voice’ strategy that talked about conversations and customer voice…