1. Introduction

With the rapid development of E-commerce livestreaming, consumers could enjoy shopping while watching videos on various livestreaming platforms. During a live broadcast, a live streamer would introduce hundreds of products, and the consumer has to find the specific product from the online-shop, which is unfriendly and laboursome. Besides, spectators may find products appearing in movies and television appealing, and want to buy them. Hence, finding products that appeared in multimedia content is desirable. However, product identification still remains a challenge in real-world applications, because of large deformations, occlusions, motion blur, the similarity of appearance between products and discrepancies across domains between videos and e-shop images. Previous public benchmarks and proposed methods are mainly focused on retrieval real-world images against e-shop images, performing product identification for multimedia content with additional temporal, textual and audio information is rarely explored. We launch the “Watch and Buy: Multimodal Product Identification” Challenge with the largest multimodal product retrieval dataset, which could help researchers further advance the performance and applicability of correspondingly built systems in real-world conditions.

2. Schedule

The Contest is in four stages: Team Registration, Preliminaries, Semifinals, and Finals. In the Preliminaries, the participating teams download the test datasets, design and debug algorithms locally, and then submit the result files to the official website. In the Semifinals, the test datasets are not visible to the contestants or available for download. Teams submit their algorithms in Docker images that are tested and graded by the system. In the Finals, teams need to reproduce their algorithms and submit their algorithms technical solution report.

Time zone: Beijing, UTC+8

• April 20, 2021 Website ready and call for registration
• 
  
• April 25, 2021 Dataset available for Download
• 
  
• May 6, 2021 Submission of Test A starts ( Season1: Preliminaries)
• Participating teams download the datasets from the official website, debug the algorithms locally, and then submit the results to the website.
• Each team can perform 3 submissions each day. After each submission, the system immediately tests and grades the algorithm. The leaderboard that is updated each hour displays the highest score of each team.
• June 20, 2021 Submission of Test A ends
• The Top 150 teams need to submit their code for review. Then Top 100 teams in season1 enroll in the season2, other teams will be eliminated.
• June 22, 2021 Submission of Test B starts (Season2: Semifinals)
• The test datasets for the Semifinals are not visible to the contestants or available for download. Teams submit their algorithms in Docker images (1 * NVIDIA V100 GPU could be used for inference). Then, the system tests the algorithms by using the test datasets and grades the algorithms. If you have no experience with Docker, click here to get started.
• Warm-up (June 22, 2021 - June 26, 2021): Participating teams can submit their algorithms and get the algorithms tested by the system 5 times each day. Each submission needs to be within 5 hours.
• Competition (June 27, 2021 - July 5, 2021): Participating teams can submit their algorithms and get tested by the system 3 times each day. Each submission needs to be within 5 hours. The leaderboard that is updated each hour displays the highest score of each team.
• Top 10 teams in season2 enroll in the season3, other teams will be eliminated.
• July 6, 2021 Algorithm reproduction starts (Season3: Finals)
• In Final season, the train dataset will be mounted in docker images for final model training, and the test datasets are not visible to the contestants or available for download. Teams need to submit their complete end-to-end code (including data processing, model training and test inference, etc.) in docker images. The submitted docker image corresponding to the best score will be directly used for code review. If the docker image code corresponding to the best score is not a complete code, it will be eliminated directly. Contestants cannot use pre-trained models of season2 directly. The final leaderboard will be determined by the reproduced results only.
• Each team will be provided a virtual machine for seven days, submission times is unlimited. The machine configuration is as follow: 4 * NVIDIA V100 GPU with 32G memory, 48 vCPU with memory 368 GB (Intel Xeon(Skylake) Platinum 8163)
• Besides, all teams need to submit their algorithms technical solution report
• July 12, 2021 Final submission deadline. Algorithms technical solution report submission deadline
• 
  
• July 20, 2021 Evaluation results announce
• The Contest committee review all codes of season3 and eliminate the teams who violate the rules. Final leaderboard will be announced after code review
• July 30, 2021 Workshop Paper submission deadline
• 
  
• Oct 20~24,2021 Workshop Starts

3. Registration to the Competition

1. The registration will start at 10:00 a.m. on April 20, 2021 UTC+8.
2. 
   
3. Watch&Buy Challenge 2021 is open to all individuals, companies, colleges, research institutions. It is a team-based contest. Each individual cannot be a member of multiple teams.
4. The registration information must be correct and valid. Any false information or cheating behavior will lead to disqualification for ranking and awarding.
5. Registration method: Participants can login Tianchi platform with the Alibaba Cloud account. Click one of the following links for registration of Alibaba Cloud: International Site, 中国站, Japanese Site.
6. For Japanses Site users, if you have trouble when you login, this post may help.
7. Members of Alibaba Group can participate in the Competition, but are excluded from the Awards. Advisory board members (and their immediate families and members of the same household) of the Competition Sponsor, Tianchi and their respective affiliates, subsidiaries, contractors, agents, judges and advertising and promotion agencies are not eligible to participate in the Competition.

4. Awards

RankTeamsAward1st place1 team$6,000 USD2st place1 team$3,000 USD3st place1 team$1,000 USD

5. Rules

• This challenge contains two leaderboards for Test A and Test B. For Test A, the scores are evaluated by the retrieval results submitted by contestants; For Test B, contestants should submit docker images at Tianchi Platform and the retrieval results is generated by running the images (We provides the VPS for running the images). You may find the docker tutorial in Tianchi Platform here.
• The final rank and score will be determined by the reproduced results on Test B ONLY. The top three performing teams will receive award certificates and cash prizes.
• One account per participant. You cannot sign up to Tianchi from multiple accounts and therefore you cannot submit from multiple accounts.
• Sharing code privately outside of teams is not allowed.
• 
  
• We ask that you respect the spirit of the competition and do not cheat. Hand-labeling is forbidden.
• 
  
• Contestants must submit a technical report that includes a summary of their submission (the standard ACM Multimedia 2021 are recommended, suggested length 1-4 pages). The reports might be made public. Only submission with the report will be considered any awards and will be put in the final leaderboard of the competition.

6. Contact

If you have any questions, please contact wab2021@service.alibaba.com

7. Reference

8. Organizer