How AI Changed the Way I Do Research and Write Research Papers

My latest paper, “Zero-Shot Confidence Estimation for Small LLMs: When Supervised Baselines Aren’t Worth Training,” is now published. It’s an empirical paper — a step toward getting back into the research world. However, this post isn’t about the paper itself — it’s about how dramatically the research process has changed since my last one.

My previous research paper was in 2020. It won the Best Student Paper Award at HOST 2020 — before ChatGPT, before LLMs became a household term. I still vividly remember what that process looked like.

Research in 2020:

We started with a proposal. Then weeks of literature review — reading paper after paper, manually tracking baselines, understanding the state of the art, mapping gaps. Then months of experiments: writing all the automation code by hand, setting up infrastructure, running trials, analyzing results, generating figures and tables, writing reports. Iteration after iteration until we had something worth publishing.

Then came the writing. English is not my first language. I’d spend days on a first draft, get feedback from advisors, rewrite, get more feedback, rewrite again. I still remember those late nights proofreading every sentence — hunting for grammar errors, awkward phrasing, anything that might undermine the work. The writing was often harder than the research itself.

Research in 2025:

For this new paper, AI was involved at nearly every step. Literature review that used to take weeks took hours — I could quickly survey related work, identify gaps, and understand where our contribution fit. Experiment code, analysis scripts, figure generation, report writing — all accelerated with AI assistance. The proofreading that used to keep me up at night? An AI catches grammar issues and awkward expressions in seconds.

To give you a sense of scale: this paper involved 3 model families, 2 datasets, ~4,500 evaluation queries, 11 experiments, and $123 in cloud costs. The entire project — from first experiment to camera-ready paper — took about 4 weeks. In 2020, a project of similar scope would have taken 4-6 months.

What hasn’t changed:

The thinking. AI accelerates execution, but the researcher still has to ask the right questions, design the right experiments, interpret the results honestly, and decide what story the data actually tells. When I found that supervised routing collapses out-of-distribution while zero-shot signals transfer, no AI told me that was the key finding — that came from understanding the problem deeply enough to recognize what mattered.

What this means for research:

More and more, each researcher — especially in CS — acts like a research advisor or project lead. You shape the direction, design the experiments, make the judgment calls. AI helps with execution. This is tremendously empowering. A single researcher can now do work that previously required a team.

But it also raises the bar. If it’s easier to produce a solid paper, then “solid” is no longer enough for top venues. The threshold for what counts as a meaningful contribution keeps rising. Breakthroughs matter more than ever, because the baseline of “competent empirical work” is now achievable by anyone with the right tools and the right questions.

I’m grateful to have experienced both eras. The 2020 paper taught me the discipline of doing everything by hand — the patience, the rigor, the attention to detail. The 2025 paper showed me what’s possible when you combine that discipline with modern tools.

The future of research isn’t AI replacing researchers. It’s researchers who know how to leverage AI outpacing those who don’t.

📄 Paper: https://arxiv.org/abs/2605.02241 💻 Code & data: github.com/BuffaloTechRider/zero-shot-llm-confidence-estimation

#Research #AI #MachineLearning #AcademicWriting #LLM #PhD