Publications

SIGNIFICANCE: There is potential for generative artificial intelligence (AI) to perform "full stack" academic publishing, including comprehensive manuscript fabrication, "peer"-review, and editorial decision making. This study evaluated the benefits and cautions in using AI for full stack academic publishing in clinical research about seizures.

METHODS: Three epileptologists each selected 1 published manuscript in Epilepsia and produced an AI-fabricated manuscript about seizures. AI tools were then used to perform "peer"-review and make editorial decisions with minimal human supervision. The rates of identifying AI-fabricated manuscripts and editorial decisions among the epileptologists was analyzed, and epileptologists' observations on how AI differed from typical human "peer"-review and editorial decisions was collected.

RESULTS: Four of 6 (67%) pairs of AI-fabricated and human-generated manuscripts were correctly identified. Only 1 AI-fabricated manuscript was editorially rejected, whereas another had a subtle indicator of fabrication. When compared to human content, AI generated content was comprehensive, detailed, and based on the highest standard of academic publication. However, generated content was more verbose, used more structured responses, and more critical feedback.

CONCLUSIONS: These results demonstrate that the quality of AI-fabricated manuscripts about clinical research in seizures was sufficiently high that the current methods of peer and editorial review could not easily distinguish fabricated content compared to human-written content. Moreover, AI review and editorial decisions were high quality and appeared reasonable in many cases. AI tools have matured to the point that there should be thoughtful policies that permit AI assistance but prohibit insufficiently supervised autonomous scientific submissions, reviews, and editorial decisions.

HIGHLIGHTS: The maturity of AI tools to assist in manuscript generation, peer-review, and editorial decisions comes with cautions.Only 1 of 3 artificial intelligence (AI) fabricated manuscripts about seizures were reliably identified and rejected.This study describes characteristics of AI-generated peer-review and editorial comments that had insufficient human oversight.

Periodic discharges (PDs) and rhythmic delta activity (RDA) are common EEG patterns in critically ill patients that require characterization of lateralization, spatial localization, and frequency. Manual characterization is subjective and time-consuming. We present an automated system for all four major subtypes: lateralized and generalized PDs (LPD, GPD) and lateralized and generalized RDA (LRDA, GRDA).&#xD;&#xD;Approach. The PD-Profiler combines a per-channel convolutional neural network, hemisphere-specific learned evidence traces, dynamic programming with an approximately periodic prior, and discharge-locked topographic localization. The RDA-Profiler uses iterative narrowband Hilbert refinement for frequency and lateralization, with phase-locking-value analysis for spatial extent. Both pipelines were trained and evaluated on 12,425 EEG segments from 11,729 patients.&#xD;&#xD;Main results. Four electroencephalographers independently scored 200 stratified segments per subtype. The algorithm exceeded expert-expert (EE) agreement on LPD frequency (expert-algorithm (EA) ICC 0.931 versus EE 0.897; p<0.001) and was statistically indistinguishable from EE on GPD frequency (EA 0.980 versus EE 0.973; p=0.061), GRDA frequency (p=0.056), LRDA frequency (p=0.435), LPD laterality (EA kappa 0.945 versus EE 0.974; p=0.079), and LRDA laterality (EA kappa 0.961 versus EE 0.993; p=0.269). No attribute fell significantly below the EE ceiling. PD discharge timing achieved an F1 of 0.817 at single-sample precision (mean absolute timing error 3.1 ms). PD spatial localization reached 97.3% of expert inter-rater Jaccard agreement, and RDA spatial extent matched EE agreement.&#xD;&#xD;Significance. GROND reaches expert-level reliability across lateralization, spatial localization, discharge timing, and frequency for PDs and RDA. In discordant cases, experts judged its frequency estimates more accurate than the original labels in 94% of cases, supporting reproducible characterization at scale.

BACKGROUND: Clinicians increasingly use AI systems to search the medical literature, but current benchmarks do not jointly test whether a response identifies the originating paper and its supporting passage. LitBench evaluates four dimensions: stated versus interpretation-requiring facts, the number and composition of competing papers, one-versus two-paper evidence, and refusal when evidence is absent.

METHODS: LitBench comprises 1,980 open-access papers (1,000 epilepsy papers holding the answers, 980 decoys from unrelated fields) and 9,472 human-reviewed facts, giving 2,188 single-paper questions and 170 requiring a fact from each of two papers. Difficulty rose by burying the answer among up to 2,000 distractors, then again over live PubMed Central. The same questions were then asked with the answering paper removed, so that refusal was the correct response. Four systems were tested: Gemma-4B, Gemma-12B, Sonnet-5, and DeepSeek-V4-Flash (refusal conditions only). Three model judges scored each answer by majority; two-paper questions counted only when both facts were found.

RESULTS: Across fixed-corpus single-paper conditions, accuracy ranged from 55.9% to 71.9% for Gemma-4B, 61.2% to 75.5% for Gemma-12B, and 90.0% to 91.3% for Sonnet-5. Similar epilepsy papers were harder than mixed candidate sets for both Gemma configurations. Two-paper accuracy was 5.7%, 17.1%, and 37.8%, respectively. With evidence absent, Gemma-12B refusal fell from 88.3% to 39.9% as candidate sets grew; Gemma-4B almost never refused. Sonnet-5 refused on 94.9% of sampled single-paper and all sampled two-paper cells. DeepSeek-V4-Flash also refused frequently, but on 80.3% of matched answer-present controls.

CONCLUSIONS: No system was reliable across every axis tested: accuracy fell as competing papers were added, interpretation-requiring facts were harder than stated ones, two-paper synthesis was rarely achieved, and only some systems recognized absent evidence. What counts as "successful" literature search carries considerable nuance; LitBench can evaluate proposed tools at a fine-grained level.