0
$\begingroup$

I'm new to tensorflow and deep-learning, I wish to get a general concept by a beginner's demo, i.e. training a (int-)number counter, to indicate the most repeated number in a set (if the most repeated number is not unique, the smallest one is chosen).

e.g. if seed=[0,1,1,1,2,7,5,3](int-num-set as input), then most = 1(the most repeated num here is 1, which repeated 3 times);

if seed = [3,3,6,5,2,2,4,1], then most = 2 (both 2 and 3 repeated most/twice, then the smaller 2 is the result)

Here I didn't use the widely used demos like image classifier or MNIST data-set, for a more customized perspective and a easier way to get data-set. so if this is not a appropriate problem for deep-learning, please help me know it.

The following is my code and apparently the result is not as expected, may I have some advice? like:

  • is this kind of problems suitable for deep-learning to solve?
  • is the network-struct appropriate for this problem?
  • is the input/output data(or data-type) right for the network?
import random
import numpy as np



para_col = 16   # each (num-)set contains 16 int-num
para_row = 500  # the data-set contains 500 num-sets for trainning
para_epo = 100  # train 100 epochs


# initial the size of data-set for training
x_train = np.zeros([para_row, para_col], dtype = int)
y_train = np.zeros([para_row, 1], dtype = int)


# generate the data-set by random
for row in range(para_row):
    seed = []
    for col in range(para_col):
        seed.append(random.randint(0,9))

    most = max(set(seed), key = seed.count)     # most repeated num in seed(set of 16 int-nums between 0~9)
    
    # fill in data for trainning-set
    x_train[row] = np.array(seed,dtype = int)   
    y_train[row] = most
    
    # print(str(most) + " @ " + str(seed))




# define and training the network

import tensorflow as tf


# a simple network according to some tutorials 
model = tf.keras.models.Sequential([
    tf.keras.layers.Flatten(input_shape=(para_col, 1)),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dense(128, activation='relu'),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(10, activation='softmax')
])


model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

# train the network
model.fit(x_train, y_train, epochs = para_epo)




# test the network
seed_test   = [5,1,2,3,4,5,6,7,8,5,5,1,2,3,4,5]
# seed_test   = [1,1,1,3,4,5,6,7,8,9,0,1,2,3,4,5]
# seed_test   = [9,0,1,9,4,5,6,7,8,9,0,1,2,3,4,5]

x_test      = np.zeros([1,para_col],dtype = int)
x_test[0]   = np.array(seed_test, dtype = int)

most_test   = model.predict_on_batch(x_test)

print(seed_test)
for o in range(10):
    print(str(o) + ": " + str(most_test[0][o]*100))

the training result looks like converged according to

...
Epoch 97/100
16/16 [==============================] - 0s 982us/step - loss: 0.1100 - accuracy: 0.9900
Epoch 98/100
16/16 [==============================] - 0s 1ms/step - loss: 0.1139 - accuracy: 0.9900
Epoch 99/100
16/16 [==============================] - 0s 967us/step - loss: 0.1017 - accuracy: 0.9860
Epoch 100/100
16/16 [==============================] - 0s 862us/step - loss: 0.1082 - accuracy: 0.9840

but the printed output looks unreasonable and random, the following is a result after one of the trainings

[5, 1, 2, 3, 4, 5, 6, 7, 8, 5, 5, 1, 2, 3, 4, 5]
0: 0.004467500184546225
1: 0.2172523643821478
2: 2.9886092990636826
3: 1.031165011227131
4: 69.71694827079773
5: 12.506482005119324
6: 1.0543939657509327
7: 0.2930430928245187
8: 8.086799830198288
9: 4.100832715630531

actually 5 is the right answer (repeated five times and most), but is the printed output indicating 4 is the answer (at a probability of 69.7%)?

$\endgroup$

1 Answer 1

0
$\begingroup$

This type of problem is not really suited to deep learning. Each node in the neural network expects numeric input, applies a linear transformation to it, followed by a non-linear transformation (the activation function), so your inputs need to be numeric. While your inputs are numbers, they are not being used numerically, as the inputs could be changed to letters or symbols. Also, your network looks like it is overfitting. It is very large for the number of inputs and so is probably just memorising the training data, which is why you appear to good results on your training data.

Tensorflow has a tensorflow-datasets package (installed separately from the main TF package) which provides easy access to a range of datasets (see https://www.tensorflow.org/datasets for details). Maybe look here to find a suitable dataset to use.

$\endgroup$
3
  • $\begingroup$ Thanks for the instruction. By what you mean While your inputs are numbers, they are not being used numerically, as the inputs could be changed to letters or symbols. , what if I normalize these random numbers to range [0.0 ~ 1.0] (as these random numbers are from 0 ~ 9, which is easily to do so, i.e. [0 ~9 ] / 9.0 ), does this make any difference or this is still a symbol-like problem? thank for further discussion. $\endgroup$
    – furynerd
    Jul 7, 2022 at 6:12
  • $\begingroup$ @furynerd: Normalising the inputs won't make any difference. The "symbol-like" nature of your inputs is an inherent property of the problem you are trying to model. $\endgroup$
    – Lynn
    Jul 7, 2022 at 11:13
  • $\begingroup$ thx for the advice, then I think I should find a more appropriate demo $\endgroup$
    – furynerd
    Jul 8, 2022 at 0:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.