How to Export Evaluation Results in Tensorflow
In TensorFlow if you’re using a tf.estimator model, for instance tf.estimator.DNNLinearCombinedClassifier, and as part of your automated training infrastructure you want to save the evaluation results as a JSON file, it’s not super straightforward, so here’s how to do it.
Let’s say you define your EvalSpec like this:
eval_spec = tf.estimator.EvalSpec(eval_input_fn,
steps=hparams.eval_steps,
exporters=[exporter],
name='eval')You’ll need to write a new exporter class that will take the eval_result from your evaluation step and save it to a file using the GFile API.
class EvalResultsExporter(tf.estimator.Exporter):
"""Passed into an EvalSpec for saving the result of the final evaluation
step locally or in Google Cloud Storage.
"""
def __init__(self, name):
assert name, '"name" argument is required.'
self._name = name
@property
def name(self):
return self._name
def export(self, estimator, export_path, checkpoint_path,
eval_result, is_the_final_export):
if not is_the_final_export:
return None
tf.logging.info(('EvalResultsExporter (name: %s) '
'running after final evaluation.') % self._name)
tf.logging.info('export_path: %s' % export_path)
tf.logging.info('eval_result: %s' % eval_result)
for key, value in eval_result.iteritems():
if isinstance(value, np.float32):
eval_result[key] = value.item()
tf.gfile.MkDir(export_path)
with tf.gfile.GFile('%s/eval_results.json' % export_path, 'w') as f:
f.write(json.dumps(eval_result))And then add that exporter to the list of exporters in your EvalSpec.
eval_spec = tf.estimator.EvalSpec(eval_input_fn,
steps=hparams.eval_steps,
exporters=[exporter, EvalResultsExporter('eval_results')],
name='eval')