Data classification is the central data-mining technique used for sorting data, understanding of data and for performing outcome predictions. In this small blog we will use a library Smilecthat includes many methods for supervising and non-supervising data classification methods. We will make a small Python-like code using Jython top build a complex Multilayer Perceptron Neural Network for data classification. It will have large number of inputs, several outputs, and can be easily extended for cases with many hidden layers. We will write a few lines of Jython code (most of our codding will deal with how to prepare an interface for reading data, rather than with Neural Network programming).
First of all, let us copy data samples. One sample will be for training, another file is for testing Copy these files to your local directory. We will import import the necessary classes to be used in this example using the usual Python style:
from smile.data import AttributeDataset,NominalAttribute
from smile.data.parser import DelimitedTextParser,IOUtils
from smile.classification import NeuralNetwork
from smile.math import Math
from jarray import zeros,array
from jhplot import *
import java
We import several classes from several packages: “smile”, “jhplot” and Java. An additional package “jarray” is used to work with Java arrays using Jython. The NeuralNetwork class is the most important for our example. It creates Multilayer perceptron neural network that consists of several layers of nodes, interconnected through weighted acyclic arcs from each preceding layer to the following. We will call it as:
nn=[Nr_input, Nr_hidden, Nr_output] # define structure of NN net=NeuralNetwork(NeuralNetwork.ErrorFunction.LEAST_MEAN_SQUARES, NeuralNetwork.ActivationFunction.LOGISTIC_SIGMOID,nn)
net.learn(x, y) # train neural net on input double array y, with the outcome given by x
Here we use logistic sigmoid function and the error function “LEAST_MEAN_SQUARES”. Before start using this NN, we will read data from the files, and create input arrays x,y. We will write a small function “getJavaArrays” that returns two arrays, double[][] and int[] (using Java style). After NN was trained, we will use the test sample and call the method “predict(x)” to verify that our prediction are close to the expected values from the test sample.
The full code looks as this:
from smile.data import AttributeDataset,NominalAttribute
from smile.data.parser import DelimitedTextParser,IOUtilsfrom smile.classification import NeuralNetwork
from smile.math import Math
from jarray import zeros,array
from jhplot import *
import java
def getJavaArrays(dataset): # this function create x[][] and y[] array from datasets
rows=dataset.size()
lst = [0.0]*rows
twoDimArr = array([lst,[]], java.lang.Class.forName('[D'))
x = dataset.toArray(twoDimArr)
y = dataset.toArray(zeros(rows, "i"))
return x,y
parser =DelimitedTextParser(); parser.setDelimiter("[t ]+")
parser.setResponseIndex(NominalAttribute("class"), 0)
train=parser.parse("Train",java.io.File("zip.train"))
x,y=getJavaArrays(train) # get input and output for training NN
print "Rescale data range .. "
p = len(x[0])
mu = Math.colMeans(x);
sd = Math.colSds(x);
for i in range(len(x)):
for j in range(p):
x[i][j] = (x[i][j] - mu[j]) / sd[j];
nin=len(x[0]); nout=Math.max(y)+1
print "Training: Nr for input layer=",nin," Nr for output layer=",nout
nn=[nin,50,nout] # 50 nodes in a hidden layer. Need another layer? Add integer after 50
print "NN layout=",nn
net = NeuralNetwork(NeuralNetwork.ErrorFunction.LEAST_MEAN_SQUARES, NeuralNetwork.ActivationFunction.LOGISTIC_SIGMOID,nn)
c1 = SPlot() # plot error vs epoch
c1.visible(); c1.setGTitle("Neural Network output error")
c1.setAutoRange(); c1.setMarksStyle('various')
c1.setConnected(1, 0); c1.setNameX('Epoch'); c1.setNameY('Error')
for j in range(70): # 70 is max value for epoch
net.learn(x, y)
error=0.0
for i in range(len(x)):
if (net.predict(x[i]) != y[i]): error +=1
error=error / len(x)
c1.addPoint(0,j,error,1)
c1.update()
print "Epoch=",j," Error=",error
print "Testing using zip.test file.."
test=parser.parse("Test",java.io.File("zip.test"))
testx,testy=getJavaArrays(test)
for i in range(len(testx)): # rescale data
for j in range(p):
testx[i][j] = (testx[i][j] - mu[j]) / sd[j];
error=0.0
for i in range(len(testx)):
if (net.predict(testx[i]) != testy[i]): error +=1
print "Error rate =", 100.0 * error / len(testx),"%"
Note that we use one hidden layer with 50 hidden nodes. The number of input (256) and outputs (10) are given by the structure of the data (looks at the data file to see their structure).
Now you can run this example using the DMelt data program (dmelt). Install DMelt, open the above code in the editor, and save it in a file with the extension “.py” (for example, as “test.py” file). This extension is important since it tells DMelt how to run this code. Then run this file: click the button with a green running man in the top menu, or press [F8]. You will see a image which shows how the learning error shrinks as a function of epoch. The last line of the code gives the error rate for your prediction.
If you are interested in deep neural networks, increase the number of hidden layers using the input list. For example, this setup has 2 hidden layers, with 50 and 25 nodes:
nn=[nin,50,25,nout]
If you will see that the number of epochs will be drastically reduced to obtain a similar (or better) rate for successful predictions (but, the CPU time, will increase since more time will be needed to adjust the weights).
