How to build an AI research portfolio employers can verify
A guide for computing researchers that the Computing Research Association published last month argues that public code repositories are becoming another form of evidence hiring teams use to size up candidates, alongside program committees reviewing paper artifacts and funders requesting data-management plans (Computing Research Association). For anyone assembling an AI research portfolio to compete for research-scientist, research-engineer, or applied-scientist roles, that raises the bar past a working demo. It suggests some reviewers now check whether a project's methods, data, and results can actually be verified, not just whether the code runs.
The guide is a field-level assessment from a professional association, not a documented survey of employer hiring practices. It doesn't establish how many companies apply this standard or how consistently, and nothing in this article should be read as proof that a given employer screens for it. What it does offer is a specific, checkable definition of reproducible work, one detailed enough that a candidate can build an AI research GitHub portfolio around it even without that proof.
What the guide means by reproducible
The CRA guide treats reproducibility as a set of habits, not paperwork tacked onto a finished result: capturing the software environment, using version control, and tracking where data came from (CRA). It also separates three claims that get lumped together. Repeatability means the original team can rerun their own setup and get the same result. Reproducibility means someone else can get the same result using the code and data the authors actually shared. Replicability means someone else gets a consistent result after independently rebuilding the method from scratch (CRA).
A project that only runs on the builder's own laptop demonstrates, at most, repeatability under the original setup. It doesn't demonstrate that anyone else can get the same result, which is the weaker of the three claims a portfolio project can support. The guide notes that a paper can be fully reproducible from a shared container and still fail to replicate, if the original finding depended on one dataset or one hardware configuration (CRA).
Some computing conferences formalize this with artifact badges for availability, functionality, and reproduced results, but those badges come from actual third-party review, not self-declaration, and they aren't a license or professional certification (CRA).
The CRA guide cites Bureau of Labor Statistics figures showing computer and information research scientists, the occupation most closely associated with "AI researcher" titles, earned a median annual wage of $140,910 in May 2024 across roughly 40,300 jobs nationally, with the occupation projected to grow 20% from 2024 to 2034 and about 3,200 openings a year over that period (CRA). That's a small, specific field worth distinguishing from the much larger population of workers whose jobs now touch some AI skill.
Building your AI research portfolio project
The CRA's criteria give candidates a practical model for documenting a project, though the guide doesn't confirm that employers actually apply those criteria when screening resumes. Treat what follows as a way to apply the standard, not as a confirmed hiring checklist.
A useful way to see how the pieces fit together: reproducing a published image-classification benchmark result on a second GPU, rather than rerunning a tutorial or repackaging a class assignment.
- Question: Can the published result be reproduced on different hardware than the original paper used?
- Evidence: a baseline comparison, the metric used and why it fits the question, a pinned environment file, and one copyable command that runs the full pipeline
- Limitation: a note on whether performance shifted with the new hardware or with different dataset preprocessing
Beyond that specific example, a repository built to this standard would generally include:
- A stated research question or claim, not just a script that runs
- Fixed random seeds, or a note on expected variation between runs
- A record of experiment runs, even a simple log
- The expected output, so someone else can tell whether their run matched
- The data source, its access terms, and license
Model cards and dataset datasheets, described in the CRA guide as community norms in machine learning, cover similar ground at the model or dataset level: intended use, training data, evaluation conditions, and known limitations for a model; provenance, composition, and appropriate uses for a dataset (CRA). Building a full one only makes sense if a candidate trained the model or assembled the dataset. For a public benchmark, linking its existing documentation is a reasonable substitute.
None of this requires reproducing a large model on a laptop. Documenting compute requirements plainly, and offering a smaller validation run or a public data subset that demonstrates the same method at lower cost, keeps the standard achievable without pretending the full-scale version is trivial to check.
What the research says about employer expectations
A study of 797 AI-related job postings on the EURAXESS platform, most from European research institutions and collected through January 5, 2024, found strong demand for research and digital skills alongside communication and interdisciplinary-collaboration skills (Frontiers in Sociology). That describes what the postings asked for, not how hiring decisions weighed those skills against technical work, and the dataset skews toward academic and research-institution listings rather than industry hiring broadly.
Demand for AI-related skills has also grown well beyond research-scientist roles specifically. Across computer and mathematical occupations, the share of job postings requiring at least one AI skill rose from 1.6% in 2010 to 12.3% in 2024 (Federal Reserve Bank of Atlanta). That figure covers a much larger occupational category than research-scientist positions, so it says something about the labor market's direction without confirming how any single employer screens research candidates.
Handling confidential work
The CRA guide focuses on public reproducibility standards and doesn't address confidential portfolio work directly. What follows is practical guidance built on the same documentation logic, not a recommendation from the guide itself.
State plainly what's restricted, the employer, the data, or both. Describe individual contribution separately from a team's output. Name what a reviewer can independently verify, a paper, a talk, a published benchmark score, and pair that with a smaller public project built to the same documentation standard. That gives a reviewer something to check even when the main work stays behind a wall.
A strong repository demonstrates research judgment and documentation discipline. It doesn't substitute for coursework, a publication record, or domain training, and it doesn't prove those things are unnecessary for a given role. That's something only the actual job posting can confirm.
Before you apply
Because the CRA guide describes an emerging practice rather than a verified hiring standard, the more useful comparison is against real postings. Pull several current listings for the exact target title, research scientist, research engineer, or applied scientist, and check what they explicitly ask for against what the portfolio project actually demonstrates. That shows whether the gap is a missing credential, a missing skill, or a missing document.
Then run the test that actually matters: hand the repository's README to someone technically competent but unfamiliar with the code, and have them install the environment, run the documented command, and report whether their output matched. Note whatever fails, fix the README, and repeat. Their result, not a self-review, is the real measure of whether the project is inspectable.
