KERAS in golang
Keras
Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. It was developed with a focus on enabling fast experimentation. Being able to go from idea to result with the least possible delay is key to doing good research.
Why use Keras?
Keras offers consistent & simple APIs, it minimizes the number of user actions required for common use cases, and it provides clear and actionable feedback upon user error. What’s more that there are several utilities in python like tqdm,matplot, numpy which empower writing complex neural network programs in utterly simple lines as well as adding support for extensive customization (GPU, Processing Queues).
Running neural networks code at core environments (Mobile, Firmware)
Writing code in python is no doubt great for prototypes, but running the same thing in production is cumbersome. Many researchers have been evaluating their options about the serving. Some are
1. Using language independent options for serving
1. Using REST API - Flask, Gunicorn, Other Http API Serving
2. Using gRPC
2. Writing ML model in universal formats and loading them in different environments
Using language independent options
Loading the model in a universal ecosystem to serve in the native language is sure a challenging task, given that it needs the service to have minimum latency and 100% availability.
Loading in the Machine Learning Model in the same Ecosystem
Serving of model in native layer environments like Mobile or Firmware level requires them to be ported in the correct formats which enforce obstruction in the choice of Neural Network Libraries. In particular, for firmware, we usually opt for Native languages like C, C++, golang for the logic. So writing the neural network API in C or golang can be challenging as it doesn’t provide extensibility as Keras does.
A middle road towards the solution
The idea is to write Neural Network code in wrappers like Keras or Tflearn and save the model in native format, so that, tfgo, the tensorflow wrapper in golang, can serve it seamlessly.
Now coming to the main part, code.
Yes, this is what you have been sulking for hours.
Code
Boston Dataset
idx CRIM ZN INDUS CHAS NOX RM AGE DIS RAD TAX PTRATIO B LSTAT
0 0.07875 45.0 3.44 0.0 0.437 6.782 41.1 3.7886 5.0 398.0 15.2 393.87 6.68
1 4.55587 0.0 18.10 0.0 0.718 3.561 87.9 1.6132 24.0 666.0 20.2 354.70 7.12
2 0.09604 40.0 6.41 0.0 0.447 6.854 42.8 4.2673 4.0 254.0 17.6 396.90 2.98
3 0.01870 85.0 4.15 0.0 0.429 6.516 27.7 8.5353 4.0 351.0 17.9 392.43 6.36
4 0.52693 0.0 6.20 0.0 0.504 8.725 83.0 2.8944 8.0 307.0 17.4 382.00 4.63
Python Code
#python 3
from __future__ import absolute_import
__author__email__ = "mrityunjay.kumar@talentica.com"
# Import Section
import keras
import tensorflow as tf
import pandas as pd
from keras import backend as K
import numpy as np
from keras_tqdm import TQDMCallback
from tensorflow.python.framework import graph_util, graph_io
# Dataset - Boston dataset
boston_housing = keras.datasets.boston_housing
(train_data, train_labels), (test_data, test_labels) = boston_housing.load_data()
# Shuffle the training set
order = np.argsort(np.random.random(train_labels.shape))
train_data = train_data[order]
train_labels = train_labels[order]
# Set the float64 as model saver dtype, it will help while serving in golang
print("Current floating format for Keras backend".format(K.floatx()))
floatx = "float64"
# force set to float64
keras.backend.set_floatx(floatx)
# re-initiate the learning phase
K.set_learning_phase(0)
# load the dataset into Pandas
column_names = ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD',
'TAX', 'PTRATIO', 'B', 'LSTAT']
df = pd.DataFrame(train_data, columns=column_names)
# normalize - Standard Normal
mean = train_data.mean(axis=0)
std = train_data.std(axis=0)
train_data = np.array(train_data)
train_data = (train_data - mean) / std
test_data = (test_data - mean) / std
# get the session from keras backend for model saver
sess = K.get_session()
# Keras model building
def build_model():
model = keras.Sequential([
keras.layers.Dense(64, activation=tf.nn.relu, name="input_start",
input_shape=(train_data.shape[1],)),
keras.layers.Dense(64, name="input_middle", activation=tf.nn.relu),
keras.layers.Dense(1, name="output_demo")
])
optimizer = tf.train.RMSPropOptimizer(0.001)
model.compile(loss='mse',
optimizer=optimizer,
metrics=['mae'])
return model
# build the model using Keras Dense API
model = build_model()
print("Input layer name {}".format(model.input))
print("Output layer name {}".format(model.output.op.name))
# Mode Summary
model.summary()
# training the model
EPOCHS = 100
# Store training stats
history = model.fit(train_data, train_labels, epochs=EPOCHS,
validation_split=0.2, verbose=0,
callbacks=[TQDMCallback()])
constant_graph = graph_util.convert_variables_to_constants(sess,
sess.graph.as_graph_def(),
[model.output.op.name])
MODEL_SAVER_DIR_NAME = "keras_op_folder"
MODEL_FILE_NAME = "output_model_name" + ".pb"
graph_io.write_graph(constant_graph, MODEL_SAVER_DIR_NAME, MODEL_FILE_NAME, as_text=False)
# close the session
sess.close()
Golang Code
// golang code for tensorflow serving
package main
import (
"bufio"
"encoding/csv"
"fmt"
tf "github.com/tensorflow/tensorflow/tensorflow/go"
"io"
"io/ioutil"
"log"
"os"
"strconv"
)
func main() {
var (
graph *tf.Graph
)
model, err := ioutil.ReadFile("keras_op_folder/output_model_name.pb")
if err != nil {
fmt.Println("Error loading saved model: %s\n ", err.Error())
return
}
graph = tf.NewGraph()
if err := graph.Import(model, ""); err != nil {
fmt.Println(err)
}
session, err := tf.NewSession(graph, nil)
if err != nil {
log.Fatal(err)
}
//defer session.Close()
// load the boston dataset here from csv
csvFile, _ := os.Open("dataset/boston/boston_keras.csv")
reader := csv.NewReader(bufio.NewReader(csvFile))
// skip first row
line, error := reader.Read()
if error != nil{
fmt.Println(error)
} else{
fmt.Println("We got rows as : ")
fmt.Println(line)
}
var index_number = 0
for {
index_number +=1
line, error := reader.Read()
if error == io.EOF {
break
} else if error != nil {
log.Fatal(error)
}
//a := make([][13]float64, 1)
a := make([][13]float64, 1)
a[0][0], err = strconv.ParseFloat(line[0], 64)
a[0][1],_= strconv.ParseFloat(line[1], 64)
a[0][2],_ = strconv.ParseFloat(line[2], 64)
a[0][3],_ = strconv.ParseFloat(line[3], 64)
a[0][4],_ = strconv.ParseFloat(line[4], 64)
a[0][5],_ = strconv.ParseFloat(line[5], 64)
a[0][6],_ = strconv.ParseFloat(line[6], 64)
a[0][7],_ = strconv.ParseFloat(line[7], 64)
a[0][8],_ = strconv.ParseFloat(line[8], 64)
a[0][9],_ = strconv.ParseFloat(line[9], 64)
a[0][10],_ = strconv.ParseFloat(line[10], 64)
a[0][11],_ = strconv.ParseFloat(line[11], 64)
a[0][12],_ = strconv.ParseFloat(line[12], 64)
var answer float64
answer,_ = strconv.ParseFloat(line[13], 64)
tensor, _ := tf.NewTensor(a)
//fmt.Println(a)
// dummy tensor for testing
//dummyArr := make([][13]float64, 1)
//fmt.Println(dummyArr)
//tensor, _ := tf.NewTensor(dummyArr)
result, err := session.Run(
map[tf.Output]*tf.Tensor{
graph.Operation("input_start_input").Output(0): tensor, // Replace this with your input layer name
},
[]tf.Output{
graph.Operation("output_demo/BiasAdd").Output(0), // Replace this with your output layer name
},
nil,
)
if err != nil {
fmt.Printf("Error running the session with input, err: %s\n", err.Error())
return
}
fmt.Printf("For Index %d : \t answer is : %f \t Result value: %v :::: \n",
index_number ,
answer,
result[0].Value().([][]float64))
}
}
Subtle Errors which are worth looking after
- Error running the session with input, err: Expects arg[0] to be double but float is provided
- The case when you have configured your keras backend as float64 but you are passing float32 as input in golang
- You must configure your input type properly while feeding to a tensorflow session as float64
- Error running the session with input, err: Expects arg[0] to be float but double is provided
- The case when you have configured your keras backend as float32 or default but you are passing float64 as input in golang
- You can use
strconvlibrary to convert your string input to float32 or float64 depending on use.
- Make sure that your input operation and output op name are consistent otherwise you will get runtime error stating incorrect layer name.
Nonetheless, I have researched a bit around available wrappers for golang. Here are some prominent libraries for Machine Learning in general.
In case of any help or suggestions, write to me [mrityunjay]dot[kumar]at[talentica.com]
~ Mrityunjay Kumar