Obtaining the Tasks:

The competition tasks will be made available at the Data & Task page on the starting date of the year CyberAI Cup competition. All participants must register their identities at the Registration page to download the datasets for having the year competition tasks.

Results Submission:

The method for results submission is via the form on the Submission page. To be ranked, submissions must include only the results for the testing datasets. Multiple submissions are allowed for one participant or team. Note that participants should submit only the class label of the testing dataset in a plain TXT format file, where the class label needs to be in the same definition as the training datasets. In addition to results submission, you are encouraged to submit a 2-8 page short paper to the AICS workshop.

Evaluation and Ranking:

The results of classification will be represented in a confusion matrix composed of tp (true positive), fn (false negative), tn (true negative), and fp (false positive), respectively. We define the precision and recall as: Precision =tp/(tp+fp), Recall=tp/(tp+fn). Then, the results are evaluated by calculating F-measure as F=2*(Precision*recall)/(Precision+Recall) and the average accuracy as:

Where n is the total number of classes, correct_i and total_i is, respectively, the correctly classified instances and the total number of instances of the ith class.

Note that only the last valid entry of result submission will be counted for performance evaluation and ranking, and a valid entry must include the results for the year all competition tasks.

Method Verification:

To prevent cheating in the CyberAI Cup, the top ranking teams will be required to fill out a fact sheet to describe their methods used for the competition. The ICSDS governing board will review the method and confirm the ranking. Note that participants might be required to provide their compiled software and/or source code.

Winner Announcement and Awarding:

The CyberAI Cup will announce the year 1st place winner at the AICS workshop, which is collocated with the International Conference on Neural Information Processing (ICONIP). The awarding of cash prize and winner certificate will be at the banquet of the ICONIP conference. The CyberAI Cup cash prize is set as normally $3000NZD, however the amount of prize will be subject to the year sponsorship grant received by the CyberAI Cup.