• Coronagraph Instrument Exoplanet Imaging Data Challenge (now closed)

• Microlensing Data Challenge #1 (now closed)
  Results

• Coronagraph Instrument Exoplanet Spectral Imaging Data Challenge (now closed)
  Results

What was the Challenge?

The Roman Exoplanet Imaging Data Challenge is for exoplanet scientists who are interested in learning the art and science of high contrast imaging of exoplanetary systems. Roman's Coronagraph Instrument, with a possible Starshade, is a high contrast exoplanet imaging instrument planned for flight in the next decade. The Data Challenge is an excellent way to become familiar with the intricacies of the first spaceborne mission with a coronagraph using active wavefront control, as a pathfinder to future flagship missions. The challenge will focus on point-source extraction (angular and reference differential imaging) and astrometry (orbital fitting).

Some of the problems that participants will face:

• Extract hidden objects around a nearby star with PSF subtraction and eventually post-processing
• Discriminate injected planet(s) from eventual background, noise, or contamination source(s) using parallax and proper motion information
• Find orbital solution(s) combining:
  • Several astrometric epochs throughout several years of the mission including both Hybrid Lyot and Starshade coronagraphs
  • Known radial velocity measurements
• Determine physical parameters of the found exoplanet(s): phase(s), albedo(s)

For more information about the data challenge events, data and results, please visit the data challenge website.

Example of three epochs (two Hybrid Lyot Coronagraph, one Starshade) for a Data Challenge 2 rehearsal and hackathon.

What were the Results?

The results of the Coronagraph Instrument Data Challenge will be announced at a public event on April 30th, 2021, from 8 - 11:30 am PDT. During this Final Jamboree, the team will announce the winners of the data challenge, the prizes, and discuss the lessons learned. The team will also describe access to the legacy tutorials and data. The finalist teams will give feedback on their techniques used to complete challenge.

Click here to register for the event. The event will be streamed live on Facebook live.

Please check the data challenge website for any updates to the event.

What was the Challenge?

WFIRST is expected to detect thousands of microlensing events, including hundreds of planetary ones. Traditionally, interesting lightcurves were modeled one-by-one given that the most exciting events were relatively rare. Such interactive modeling will not be possible as the data set grows, so to fully exploit this dataset, analysis techniques need to be upgraded.

To stimulate research in this area, there will be a series of microlensing data challenges, the first of which is based around a large set of simulated WFIRST lightcurves. This dataset has now been released, with a submission deadline of Oct 31, 2018.

For more information, please visit http://microlensing-source.org/data-challenge/ .

There is also a python notebook tutorial to provide a starting point for newcomers wishing to take part.

What were the Results?

A total of 293 light curves in filters Z087 and W149 were simulated by Matthew Penny, and this included 74 single lenses (including free floating planet candidates), 83 binary star lenses, 43 planetary binary lenses, and 93 cataclysmic variables. The simulated light curves mimicked the cadence, length, and noise properties of the multi-year WFIRST Bulge survey.

A sample lightcurve mimicking the expected cadence and noise characteristics of WFIRST microlensing observations

Four teams entered the challenge, including 16 participants in total, seven of whom were newcomers to the microlensing field. Teams were tasked with classifying each light curve and providing fits to the model parameters of each. Teams were evaluated on the accuracy of their fitted parameters, the efficiency and scalability of their software and modeling approaches, innovations they brought to the table, and the extent to which newcomers participated.

All teams used publicly available software, and progress was made on the issue of scalability, although there is still room for improvement. Some new approaches to classification and detection were developed but are still in their early stages. Results from each of the teams showed that, when microlensing events were properly classified, parameters were also accurately derived, modulo some known weaknesses such as a tendency to overestimate the impact parameter. However, the problem of classification is non-trivial, particularly for subtle anomalies in the light curves. Formal benchmarking, which was not attempted in this challenge due to logistical reasons, would have provided a more meaningful comparison between teams. More details can be found in this slide package.

Written feedback was sent to each team, and a paper documenting the challenge and its results is now being written. A second data challenge, building on the lessons learned here, will soon follow.

What was the Challenge?

Welcome to the inaugural Coronagraph Instrument Exoplanet Data Challenge! The WFIRST mission is currently in Phase A, during which time the science and instrument performance requirements will be defined for exoplanet imaging and spectroscopy. In order to provide the project with the best possible inputs before the end of Phase A in 2017, we are seeking participation from teams with spectral retrieval expertise through the data challenge.

The Challenge will run from August 2016 to March 2017. The 2016 Challenge consists of a blind spectral retrieval exercise using simulated, extracted spectra for several known RV and/or hypothetical discovery exoplanets. The spectra will NOT need to be extracted from simulated IFS data. Instead, we will explore the impact of signal-to-noise ratio and spectral resolution on the detection/measurement of atmospheric abundances and other planet properties. Even with that relatively simple goal, we expect the Challenge to be non-trivial!

Sample spectra from the Data Challenge. Shown are simulated spectra for three of the four exoplanets at the highest signal-to-noise and resolving power (top row) and at the lowest signal-to-noise and resolving power (bottom row).

Incentive to Participate: While defining the first space-borne exoplanet imaging mission is hopefully its own compelling reason for doing this, to make this a little more fun the Coronagraph Instrument Exoplanet Spectral Imaging Data Challenge Science Investigation Team is offering travel expenses and registration costs for one person on each team that fully completes the Challenge (all four planets, all SNR and R values, all requested retrieval outputs) to attend the 2017 WFIRST Science Meeting, or another exoplanets meeting of his/her choice (up to $2000).

Participation in the Challenge is contingent upon acceptance of terms which will be included in the invitation email.

If you wish to participate, please register and you will be sent an invitation.

If you have questions, please forward them to Margaret Turnbull and David Ciardi through the "Contact" link above.

We look forward to working with you this Fall!

What were the Results?

The Coronagraph Instrument Exoplanet Spectral Imaging Data Challenge is a working group within the SIT "Harnessing the Power of the WFIRST-Coronagraph" (PI, Margaret Turnbull). The goal is to provide a quantitative and qualitative feedback to the teams defining the specifications and requirements of the WFIRST coronagraph by blindly retrieving realistic estimates of the expected planetary and atmospheric properties that can be recovered in the WFIRST mission.

Renyu Hu, Tyler Robinson, and Jake Lustig-Yaeger produced simulated data representative of the Integral Field Spectrograph (IFS) instrument of WFIRST. The simulations covered different types of planets (Hot giants, Sub-Neptunes, and rocky ones). They included different instrumental modes of the WFIRST IFS and covered a representative range of different signal-to-noise scenarios. They had different concentrations of CH₄, NH₃, an H₂O in their atmospheres, as well as different temperatures and masses. In this Data Challenge, we considered two giant planets, one sub-Neptune, and two rocky planets.

There were numerous participants from institutions in Asia, Europe, and the US, with several members outside the WFIRST community, and we held regular telecon meetings to share our results.

One important outcome from the Data Challenge was the cross-comparison of the atmospheric modeling among the different teams, significantly increasing the agreement among them, while still leaving room for a variety of atmospheric models. Some teams were able to recover atmospheric parameters.

Two teams, NEMESIS (P. Irwin, J. Eberhardt, and R. Garland) and the team led by M. Marley, R. Lupu, and M. Nayak, were able to recover the atmospheric parameters for the first two giant planets including a variety of models with clouds and hazes. Our major conclusion is that CH₄ concentration was well recovered for the WFIRST data. NH₃ and H₂O recovery was acceptable. The latter was highly dependent on the signal-to-noise ratio and absorption tables used to derive the results. Finally, fundamental parameters, such as the mass of the planet, had a high statistical precision but were somewhat biased depending on the atmospheric model.

The figure above shows the results from two teams. Left: Data and retrieved spectra (credit M. Nayak). Right: Recovered atmospheric parameters for one gas giant (credit: R. Garland).