Lecture VIII: Your Research Project

AI-Assisted Programming for PhD Researchers

Author

Affiliation

Dr. Tobias Vlćek

Helmut Schmidt University, August 2026

Research Code Is Different

The Code Is the Claim

In research, code correctness is result validity.

. . .

A bug here is not a crash but a wrong Table 1
Remember Lab 3: one silent fillna shifted every summary statistic
Nobody sees a stack trace; they see a number that looks fine and is not
The code and the result stand or fall together

Reproducibility Checklist

Four things make a result someone else can rerun:

Pinned environment: uv lock, renv, or Project.toml
Fixed seeds: every random draw is deterministic
Data versioned or archived: findable and persistent (Wilkinson et al. 2016)
One command from raw data to every figure

Notebooks: Handle with Care

Hidden state and out-of-order execution quietly wreck reproducibility
Of ~1.16M public notebooks Pimentel et al. mined, only ~4% reproduced their own saved output (Pimentel et al. 2019)
36% ran cells out of order (a contributing cause, not the failure rate itself) (Pimentel et al. 2019)
Fine for exploration; the record should be a pipeline, like the one you built

Agents and Reproducibility

Agents are good at exactly this cleanup work:

“Make this run end-to-end from scratch in a clean environment”
“Pin my dependencies and generate a lock file”
“Find the hidden state in this notebook and flag out-of-order cells”

Zed’s Built-in REPL

Zed can run code inline, cell by cell, in an ordinary .py file, no separate notebook file at all
Backed by Jupyter kernels. Same execution model as Jupyter, but your code stays an ordinary script: no saved outputs, nothing a .ipynb can hide
The kernel needs to be installed in your project’s environment first: uv add --dev ipykernel, then repl: refresh kernelspecs
Worth exploring after the course for fast exploratory analysis

Remote Development over SSH

Zed can edit and run code directly on a remote machine (a cluster, a lab workstation) over SSH
The UI stays local and responsive; language servers, terminals, and tasks all run on the remote
Connect via Remote Projects (ctrl-cmd-shift-o / alt-ctrl-shift-o) or zed ssh://user@host/path
Relevant if your research runs on HPC, worth exploring after the course

AI Beyond Code

Literature Research with Agents

The deep-research pattern: search → read → synthesize with citations.

. . .

Point an agent at a question; it fans out, reads, and summarizes with sources
Genuinely useful for mapping an unfamiliar field fast
A strong starting map, not a finished bibliography

The Catch: Invented Citations

Models produce references that are confident, well-formatted, and fake:

55% of GPT-3.5’s citations were fabricated; GPT-4 still 18% (Walters and Wilder 2023)
Even the real ones carried errors: 43% (GPT-3.5) / 24% (GPT-4) (Walters and Wilder 2023)
Newer is better, not safe: GPT-4o still ~20% fabricated, worst on niche topics (Linardon et al. 2025)

Spot the Fake

Same format, same confidence. One exists, one does not:

Fabricated: Smith, J. (2019). "Deep learning for
  epidemic forecasting." Nature Methods, 16(4), 331–339.

Real:       Pimentel et al. (2019). "A large-scale study
  about quality and reproducibility of Jupyter notebooks."
  MSR 2019, 507–517.

You cannot tell from the text, so every citation gets verified.

The Verification Discipline

For every AI-suggested source, three checks:

DOI resolves: the link actually goes somewhere
Paper exists: real venue, real authors, real year
It says what was claimed: read the abstract, minimum

. . .

AI does not remove the work; it moves it: from finding to verifying (Lee et al. 2025). This afternoon’s Lab 5 rule, now for your thesis.

Drafting Text with Skills

A clunky sentence through conciseness + academic-grammar:

Before: It should be noted that, in order to obtain the
  results, we made use of a large number of samples.

After:  We used a large sample.

The skills suggest; you decide what stays.

Cross-Check Everything

AI drafts are drafts: verify claims sentence by sentence, numbers against sources, citations against papers.

. . .

The deeper risk is producing more while understanding less: “illusions of understanding” (Messeri and Crockett 2024). You sign it, you own it.

Institutions Are Responding

AI agents in research are now a policy topic: journals, funders, and institutions are writing the rules (“AI Scientists Are Changing Research — Institutions, Funders and Publishers Must Respond” 2026)
Provider watermarks (Lecture VI) hand journals a detection tool, so expect checks to become routine
The disclosure and verification habits from Lecture VI already keep you ahead: the requirements are only catching up

Your Capstone

The Assignment

Tomorrow you work on your project. This afternoon you plan.

. . .

Take a real problem from your own research and point an agent at it
Tomorrow afternoon, everyone shows what they built and what went wrong
No slide deck needed. The working thing and an honest account of it are enough

What Makes a Good Capstone

A real problem from your own research
The core fits in ~2–3 hours; anything bigger goes in stretch goals
Something checkable comes out: a figure, a table, a test suite that passes
Runs the whole workflow: spec → plan → implement → verify

No Project? Option A

A BibTeX/DOI tool built from the Lab 5 pieces:

Parse a messy .bib file or a list of DOIs
Fetch metadata via the reference-lookup skill
Drop the duplicates and write out a clean reading list
Full brief on the capstone page

No Project? Option B

A small simulation with a parameter sweep:

Pick a model: a queue, an epidemic, whatever fits your field
Sweep two parameters over a grid
Plot the results and put the numbers in a table
Full brief on the capstone page

Scope It Now

Open the capstone page and write your spec with the interview pattern
By 16:30: spec + plan reviewed by a fresh agent session

The Capstone Project

Continue Your Journey

Next Up

You now know why research code carries a higher bar, and how to keep AI honest in it
Tomorrow, 09:00: we build.
The Capstone Project: scope yours this afternoon
Lecture IX: Judgment and Showing Your Work, tomorrow morning
Course literature and references

References

“AI Scientists Are Changing Research — Institutions, Funders and Publishers Must Respond.” 2026. Nature 651 (8107): 853–54. https://doi.org/10.1038/d41586-026-00934-w.

Lee, Hao-Ping (Hank), Advait Sarkar, Lev Tankelevitch, et al. 2025. “The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers.” Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (Yokohama Japan), April, 1–22. https://doi.org/10.1145/3706598.3713778.

Linardon, Jake, Hannah K Jarman, Zoe McClure, Cleo Anderson, Claudia Liu, and Mariel Messer. 2025. “Influence of Topic Familiarity and Prompt Specificity on Citation Fabrication in Mental Health Research Using Large Language Models: Experimental Study.” JMIR Mental Health 12 (November): e80371. https://doi.org/10.2196/80371.

Messeri, Lisa, and M. J. Crockett. 2024. “Artificial Intelligence and Illusions of Understanding in Scientific Research.” Nature 627 (8002): 49–58. https://doi.org/10.1038/s41586-024-07146-0.

Pimentel, Joao Felipe, Leonardo Murta, Vanessa Braganholo, and Juliana Freire. 2019. “A Large-Scale Study About Quality and Reproducibility of Jupyter Notebooks.” 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR) (Montreal, QC, Canada), May, 507–17. https://doi.org/10.1109/MSR.2019.00077.

Walters, William H., and Esther Isabelle Wilder. 2023. “Fabrication and Errors in the Bibliographic Citations Generated by ChatGPT.” Scientific Reports 13 (1): 14045. https://doi.org/10.1038/s41598-023-41032-5.

Wilkinson, Mark D., Michel Dumontier, IJsbrand Jan Aalbersberg, et al. 2016. “The FAIR Guiding Principles for Scientific Data Management and Stewardship.” Scientific Data 3 (1): 160018. https://doi.org/10.1038/sdata.2016.18.