arXiv daily digest

Pathfinder is now published in ApJS!

One of the things I wanted to do with this framework is to keep track of the literature in an easily digestible form. As a first step to this, we can create a daily digest for each of the different arXiv astro-ph categories, that you can find here:

• Astrophysics of galaxies (astro-ph.GA)
• Cosmology and nongalactic astrophysics (astro-ph.CO)
• Earth and planetary astrophysics (astro-ph.EP)
• High energy astrophysicsl phenomena (astro-ph.HE)
• Instrumentation and methods in astrophysics (astro-ph.IM)
• Solar and stellar astrophysics (astro-ph.SR)

The way this works is pretty straightforward. We scrape arXiv.org for papers published on a given day, use pathfinder to find similar papers for each new paper (so that we can get background and context), make one LLM call per paper to distill what is new and novel about the current paper in context of current literature, and then one final LLM call to take all this information and distill it down into one reasonably concise summary.