Directions for participating in the WASSA-2017 Emotion Intensity Shared task via CodaLab


  1. Create an account in CodaLab ( Sign in.

  2. Edit your profile appropriately. Make sure to add a team name, and enter names of team members. (Go to "Settings", and look under "Competition settings".)

  3. Proceed to task webpage on CodaLab. A direct link to the Emotion Intensity CodaLab competition is here. Read information on all the pages.

  4. Download data: training, development, and test (when released)

  5. Run your system on the data and generate a submission file. A valid submission file for CodaLab is a zip compressed file containing the following files:

    Each of these files must follow the official submission format. CodaLab does not place any restrictions on the name of the zip file name. Note: Do not submit a zip compressed folder or directory, that is, do not submit a zip of a folder that includes the four files.

  6. Make submissions on the development set (Phase 1).
  7. Once the evaluation period begins, you can make submissions for the test set (Phase 2). The procedure is similar to that on the dev set. These differences apply:


Q. Why restrict the number of official submissions to one?

A. Since this is a competition, we do not want teams to submit a large number of submissions using different parameters and systems without being confident which will work best.

Even though the number of official submissions is restricted to one, the gold data will be released soon after the evaluation period. Thus you can use it to determine results from many different system variants. You are strongly  encouraged to report these additional results in the system-description paper in addition to the official submission results.

Q. Can I be involved in more than one team? (And thus make more than one official submission on the test set?

A. No. This can be used to abuse the one-official submission rule, which can be unfair to other participants.

Q. How do I include more than one score on the leaderboard?

A. CodaLab allows only one score on the leaderboard per user.