I am working on a multi-label image classification problem with the evaluation being conducted in terms of F1-score between system predicted and ground truth labels.

Given that, should I use loss="binary_crossentropy" or loss=keras_metrics.f1_score() where keras_metrics.f1_score() is taken from here: https://pypi.org/project/keras-metrics/? I am a bit confused because all of the tutorials I have found on the Internet regarding multi-label classification are based on the binary_crossentropy loss function, but here I have to optimize against F1-score.

Furthermore, should I set metrics=["accuracy"] or maybe metrics=[keras_metrics.f1_score()] or I should left this completely empty?

Based on user706838 answer ...

use the f1_score in https://www.kaggle.com/rejpalcz/best-loss-function-for-f1-score-metric

import tensorflow as tf
import keras.backend as Kdef f1_loss(y_true, y_pred):tp = K.sum(K.cast(y_true*y_pred, 'float'), axis=0)tn = K.sum(K.cast((1-y_true)*(1-y_pred), 'float'), axis=0)fp = K.sum(K.cast((1-y_true)*y_pred, 'float'), axis=0)fn = K.sum(K.cast(y_true*(1-y_pred), 'float'), axis=0)p = tp / (tp + fp + K.epsilon())r = tp / (tp + fn + K.epsilon())f1 = 2*p*r / (p+r+K.epsilon())f1 = tf.where(tf.is_nan(f1), tf.zeros_like(f1), f1)return 1 - K.mean(f1)