Artificial intelligence based personalized student feedback system 'Sisu Athwala' to enhance exam performance of medical undergraduates
A feedback system for medical undergraduates, built on retrieval augmented generation over a custom knowledge base to reduce inaccurate answers from the underlying model. It gives each student feedback on their MCQ and SAQ performance, along with guidance on stress management and study strategy. It was evaluated both by expert student mentors and by the students who used it.
Reported study results
15 expert student counsellors evaluated the feedback; 25 students reported on its usefulness.
Related capability: Retrieval-augmented generation
Read the full articleArtificial intelligence assisted automated short answer question scoring tool shows high correlation with human examiner markings
A scoring tool that uses a large language model to mark short answer questions. It extracts the key components of a student's answer, applies the scoring rubric supplied by the instructor, and writes individual feedback. It was tested on a systematic pharmacology course, where its marks correlated closely with those of human examiners.
Reported study result
Correlation reported between AI scores and scores from two human examiners.
Related product: AES
Read the full articleAligning large language models for clinical tasks
An alignment method for medical question answering, called expand-guess-refine, which combines instruction tuning with in-prompt techniques. It reached 70.63% accuracy on a subset of the USMLE dataset without additional parameters or training data.
Reported benchmark result
Accuracy reported on the study's selected USMLE dataset subset.
Related capability: Clinical AI alignment
Read the full articleRetrieval augmented generation and representative vector summarization for large unstructured textual data in medical education
Sets out retrieval augmented generation as a way to attach a customisable knowledge base to a language model, so that answers in a medical context can be grounded in a known source rather than in the model's own memory. It also presents a combined extractive and abstractive summarisation method for large unstructured text, using representative vectors.
Method overview
The paper's indexing and retrieval workflow, shown in sequence.
Related capability: Retrieval-augmented generation
Read the full article
Authors are listed in publication order. Entries marked with a university affiliation were carried out with collaborators at that institution.