报错内容：
花了一天时间，经历重装驱动到 CUDA 到系统到虚拟环境，最终最确定是因为 keras 版本问题。我其他两台机器的版本是 2.2.4，出问题这台电脑是 2.3.1，版本降到 2.2.4 就不报这个错误了。
我是第一个遇到这个问题的人，至少之前在网上根本搜不到相关问题的帖子。
报错内容：
花了一天时间，经历重装驱动到 CUDA 到系统到虚拟环境，最终最确定是因为 keras 版本问题。我其他两台机器的版本是 2.2.4，出问题这台电脑是 2.3.1，版本降到 2.2.4 就不报这个错误了。
我是第一个遇到这个问题的人，至少之前在网上根本搜不到相关问题的帖子。
Epoch 1/1
2020-01-13 21:39:19.392806: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally
2020-01-13 21:39:22.432074: W tensorflow/core/framework/op_kernel.cc:1401] OP_REQUIRES failed at tensor_array_ops.cc:122 : Not found: Resource __per_step_10/_tensor_arraysinput_ta_0_4/N10tensorflow11TensorArrayE does not exist.
Traceback (most recent call last):
File "/home/hans/WorkSpace/client.py", line 90, in <module>
hist = client.train(convertDict(dictItem), dictItem.get(f"DirName"), dictItem.get(f"round"), CRNNconfig["ESTIMATION"]["SAVE_CLIENT_MODEL"])
File "/home/hans/WorkSpace/FLutils/client.py", line 44, in train
verbose=1)
File "/home/hans/WorkSpace/venv/Rand2AI/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/home/hans/WorkSpace/venv/Rand2AI/lib/python3.6/site-packages/keras/engine/training.py", line 1732, in fit_generator
initial_epoch=initial_epoch)
File "/home/hans/WorkSpace/venv/Rand2AI/lib/python3.6/site-packages/keras/engine/training_generator.py", line 220, in fit_generator
reset_metrics=False)
File "/home/hans/WorkSpace/venv/Rand2AI/lib/python3.6/site-packages/keras/engine/training.py", line 1514, in train_on_batch
outputs = self.train_function(ins)
File "/home/hans/WorkSpace/venv/Rand2AI/lib/python3.6/site-packages/tensorflow/python/keras/backend.py", line 3076, in __call__
run_metadata=self.run_metadata)
File "/home/hans/WorkSpace/venv/Rand2AI/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1439, in __call__
run_metadata_ptr)
File "/home/hans/WorkSpace/venv/Rand2AI/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: Resource __per_step_10/_tensor_arraysinput_ta_0_4/N10tensorflow11TensorArrayE does not exist.
[[{{node training/Adadelta/gradients/replica_0/model_1/rnn1_bgru1/while_1/TensorArrayReadV3_grad/TensorArrayGrad/TensorArrayGradV3}}]]
[[{{node training/Adadelta/gradients/b_count_26}}]]