10

Low accuracy in Deep Neural Network with Tensorflow

 3 years ago
source link: https://www.codesd.com/item/low-accuracy-in-deep-neural-network-with-tensorflow.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Low accuracy in Deep Neural Network with Tensorflow

advertisements

I am following the third Jupyter notebook on Tensorflow examples.

Running problem 4, I tried to implement a function which builds automatically a number of hidden layers, without manually coding the configuration of each layer.

However, the model runs providing very low accuracy (10%) so I thought that maybe such function could not be compatible with the graph builder of Tensorflow.

My code is the following:

def hlayers(n_layers, n_nodes, i_size, a, r=0, keep_p=1):

  for i in range(n_layers):
    if i > 0:
      i_size = n_nodes
    w = tf.Variable(tf.truncated_normal([i_size, n_nodes]), name=f'W{i}')
    b = tf.Variable(tf.zeros([n_nodes]), name=f'b{i}')
    pa = tf.nn.relu(tf.add(tf.matmul(a, w), b))
    a = tf.nn.dropout(pa, keep_prob=keep_p, name=f'a{i}')
    r += tf.nn.l2_loss(w, name=f'r{i}')

  return a, r

batch_size = 128
num_nodes = 1024
beta = 0.01

graph = tf.Graph()
with graph.as_default():

  # Input data. For the training data, we use a placeholder that will be fed
  # at run time with a training minibatch.
  tf_train_dataset = tf.placeholder(
    tf.float32,
    shape=(batch_size, image_size * image_size),
    name='Dataset')
  tf_train_labels = tf.placeholder(
    tf.float32,
    shape=(batch_size, num_labels),
    name='Labels')
  tf_valid_dataset = tf.constant(valid_dataset)
  tf_test_dataset = tf.constant(test_dataset)

  keep_p = tf.placeholder(tf.float32, name='KeepProb')

  # Hidden layers.
  a, r = hlayers(
    n_layers=3,
    n_nodes=num_nodes,
    i_size=image_size * image_size,
    a=tf_train_dataset,
    keep_p=keep_p)

  # Output layer.
  wo = tf.Variable(tf.truncated_normal([num_nodes, num_labels]), name='Wo')
  bo = tf.Variable(tf.zeros([num_labels]), name='bo')
  logits = tf.add(tf.matmul(a, wo), bo, name='Logits')
  loss = tf.reduce_mean(
    tf.nn.softmax_cross_entropy_with_logits(
      labels=tf_train_labels, logits=logits))

  # Regularizer.
  regularizers = tf.add(r, tf.nn.l2_loss(wo))
  loss = tf.reduce_mean(loss + beta * regularizers, name='Loss')

  # Optimizer.
  optimizer = tf.train.GradientDescentOptimizer(0.5).minimize(loss)

  # Predictions for the training, validation, and test data.
  train_prediction = tf.nn.softmax(logits)

  a, _ = hlayers(
    n_layers=3,
    n_nodes=num_nodes,
    i_size=image_size * image_size,
    a=tf_valid_dataset)
  valid_prediction = tf.nn.softmax(tf.add(tf.matmul(a, wo), bo))

  a, _ = hlayers(
    n_layers=3,
    n_nodes=num_nodes,
    i_size=image_size * image_size,
    a=tf_test_dataset)
  test_prediction = tf.nn.softmax(tf.add(tf.matmul(a, wo), bo))

num_steps = 3001

with tf.Session(graph=graph) as session:
  tf.global_variables_initializer().run()
  print("Initialized")
  for step in range(num_steps):
    # Pick an offset within the training data, which has been randomized.
    # Note: we could use better randomization across epochs.
    offset = (step * batch_size) % (train_labels.shape[0] - batch_size)
    # Generate a minibatch.
    batch_data = train_dataset[offset:(offset + batch_size), :]
    batch_labels = train_labels[offset:(offset + batch_size), :]
    # Prepare a dictionary telling the session where to feed the minibatch.
    # The key of the dictionary is the placeholder node of the graph to be fed,
    # and the value is the numpy array to feed to it.
    feed_dict = {
      tf_train_dataset : batch_data,
      tf_train_labels : batch_labels,
      keep_p : 0.5}
    _, l, predictions = session.run(
      [optimizer, loss, train_prediction], feed_dict=feed_dict)
    if (step % 500 == 0):
      print("Minibatch loss at step %d: %f" % (step, l))
      print("Minibatch accuracy: %.1f%%" % accuracy(predictions, batch_labels))
      print("Validation accuracy: %.1f%%" % accuracy(
        valid_prediction.eval(), valid_labels))
  print("Test accuracy: %.1f%%" % accuracy(test_prediction.eval(), test_labels))

Weight regularization is stronger with more layers. Therefore you could try to reduce the regularization and see if the accuracy increases.


report

Recommend

About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK