Evaluation Server Not Working

[WhiteButterflies]

The evaluation link：https://eval.ai/web/challenges/challenge-page/1805/my-submission

But status still 'Submitted' not 'Finished'.

[HamadYA]

I am facing the same issue. Did you find a solution?

The evaluation link：https://eval.ai/web/challenges/challenge-page/1805/my-submission

But status still 'Submitted' not 'Finished'.

[WhiteButterflies]

I am facing the same issue. Did you find a solution?

The evaluation link：https://eval.ai/web/challenges/challenge-page/1805/my-submission
But status still 'Submitted' not 'Finished'.

No. May be error in Eval AI platform.

[PJD-WJ]

same question +1

[SilvioGiancola]

EvalAI went from free to paid plans, forcing the organizers of challenges to pay extensive monthly fees (125USD/month). We don't have the budget for such expense and are investigating alternative solutions.

[minwoogood12]

When would it be possible to submit the EvalAI server?

[Krisnon]

@SilvioGiancola Sorry to hear that. Have there been any updates or a timeline regarding the alternative solutions you’re investigating?

[kun-dragon]

You may publish the ground-truth of the test dataset if there are no other solutions.

[Sourabh9468]

If this issue has been resolved, kindly update the status and provide the instructions to evaluate our tracker.

[HamadYA]

EvalAI went from free to paid plans, forcing the organizers of challenges to pay extensive monthly fees (125USD/month). We don't have the budget for such expense and are investigating alternative solutions.

Sorry for disturbing.
Do you have any updates, please?

I would also suggest the following.

You may publish the ground-truth of the test dataset if there are no other solutions.

[SilvioGiancola]

I setup a new evaluation server here: https://www.codabench.org/competitions/13665/

Please stress test it, give me your feedbacks and I'll improve it.

Here is an example of a submission (STAPLE_CA), make sure the zipped file of your predictions does not contains any intermediate folder. predictions.zip

[WhiteButterflies]

I setup a new evaluation server here: https://www.codabench.org/competitions/13665/我在这里设置了一个新的评估服务器：https://www.codabench.org/competitions/13665/

Please stress test it, give me your feedbacks and I'll improve it.请对它进行压力测试，给我你的反馈，我将改进它。

Here is an example of a submission (STAPLE_CA), make sure the zipped file of your predictions does not contains any intermediate folder. predictions.zip下面是一个提交（STAPLE_CA）的示例，请确保预测的压缩文件不包含任何中间文件夹。predictions.zip

结果文件处理的很快，跟过去的EvalAI的评测结果没什么区别

[HamadYA]

I setup a new evaluation server here: https://www.codabench.org/competitions/13665/

Please stress test it, give me your feedbacks and I'll improve it.

Here is an example of a submission (STAPLE_CA), make sure the zipped file of your predictions does not contains any intermediate folder. predictions.zip

It works, but I could only register with my university email. I could not get the confirmation emails for my personal emails.

[HamadYA]

I setup a new evaluation server here: https://www.codabench.org/competitions/13665/

Please stress test it, give me your feedbacks and I'll improve it.

Here is an example of a submission (STAPLE_CA), make sure the zipped file of your predictions does not contains any intermediate folder. predictions.zip

I ran a different experiment that is in "running" status since yesterday.

[SilvioGiancola]

Codabench was down last week, it seems unstable from time to time. The processing should not take more than a few minutes (the actual time to process a submission is only a few seconds). If that happens again, please try submitting again.