Six ways to ask AI
for meeting minutes

All results

We ran each of the six ways of asking once per meeting and model: 30 runs per way of asking. Every answer was checked item by item against an answer key written before the run.

• Fully correct: every decision, condition on a decision, task owner and deadline, and open item right, no trap taken, and nothing added
• Additions: things not said in the meeting, written as fact. Anything labeled as a guess or “to confirm” was not counted
• Deadline left as said: written the way it was said, such as “next Friday” or “tomorrow,” with no date. Not wrong, but a later reader can't tell the date, so as decided before the run we did not count it as correct

The 10 meetings had 42 tasks in all, so one way of asking across three models makes 126 task deadlines (252 for one model across six ways). The deadline column counts how many of them were left as said. Additions are a count of items; the number in brackets is how many answers had at least one. Length is the number of Japanese characters in one set of minutes; for the two-step request, only the minutes from the second step.

What we did

We wrote 10 fictional meetings: a sweet shop planning Christmas cakes, an app team planning a sprint, a condominium board, a PTA bazaar committee, a publisher's production meeting, a tourism office's social media meeting, an office move, a university hiking club's winter trip, library storytelling volunteers, and a company study session. Each was about 1,300–2,400 Japanese characters, with updates and small talk mixed in.

Each meeting had moments where minutes easily go wrong:

• A decision changed near the end (a start date moved from November 1 to November 8)
• A number corrected mid-sentence (“300 yen each… no, 500 yen”)
• A task that changes hands, or one meant for someone who is absent
• Deadlines like “next Friday” or “a week before orders open,” and tasks with no deadline
• Conditional decisions (“switch if the quote is the same or lower,” “if the general meeting approves”)
• Jokes (“How about I deliver them dressed as Santa?”)
• Rough automatic transcription (speakers shown only as numbers, “reel video” turned into “rule video,” dropped audio)

There were six ways of asking, each placed before the transcript. The full Japanese text is at the end of this article.

• Plain: “Make minutes from the following meeting transcript.”
• Headings: the sentence above, plus five headings to use (meeting name, date and attendees / decisions / tasks with owner and deadline / open items / other)
• Secretary role: “You are an excellent secretary,” asked for minutes to submit to a boss
• Quote the source: adds “Don't write anything not said in the meeting. For decisions and tasks, briefly quote the remark they are based on.”
• Two steps: first list what was decided, the tasks and what was left undecided, then turn that into minutes
• Headings + quotes: adds both the headings and the two sentences about quoting

Each request was a fresh conversation, with search and other tools off. For the two-step request, only the second-step minutes were judged.

The decisions came out right

Only 1 run in 180 left out a decision (647 of 648 decisions written). All 756 task owners were right, and all 180 runs wrote the open items as still open. No run wrote an overturned decision, a number before its correction, or a joke as something decided.

Conditional decisions kept their conditions. The condominium board decided to “recommend Company B and put it to the general meeting.” No answer wrote that Company B had been chosen; all said the general meeting would decide.

The one miss was Haiku with the two-step request, which wrote only that the egg supplier “depends on the quote,” dropping the actual rule: switch if the quote is the same or lower than now.

In the pilot, with three easier meetings (45 runs), the decisions were also all correct. Longer, more confusing meetings made no difference here.

The difference: additions

Opus added nothing; Sonnet added 6 things and Haiku 16. By way of asking, the secretary role had the most, 9; plain had 5; the other four had 2 each. The pilot pointed the same way: most additions came from Haiku and the secretary role.

Most additions were small slips: a reason gotten wrong, someone's words slightly changed, a task nobody assigned. The answers were in Japanese; these are our translations.

In the tourism office meeting, Haiku made the same slip with three ways of asking: it gave the reason for not going to four posts a week as the follower count. The reason given in the meeting was not having enough staff.

We drew the line finely. Writing “already reserved” for “I'll reserve it” counted as one. Treat the counts as a rough guide.

Some owners were unclear but could be guessed from context. In the sweet shop meeting, the owner said “Work out the cost and we'll decide next time,” and the clerk who suggested the idea answered “Got it.” Three answers (Opus 2, Sonnet 1) stated flatly that this clerk would work out the cost. We counted these separately from additions.

Asking for quotes

Asking “Don't write anything not said in the meeting” and “briefly quote the remark” kept additions low: 2 in 30 runs. But deadlines were often left as said, such as “by next Friday” or “tomorrow”: 29 of 126, against 0–6 for the other ways of asking. Of Sonnet's 22 deadlines left as said, 17 came with this request.

The date can be worked out from the meeting date and the wording, but the model may have treated a worked-out date as something “not said in the meeting.” Combined with headings, deadlines left as said dropped to 6. Both points are our guesses; we did not test them.

This is almost entirely why this request had only 15 fully correct runs in 30.

Transcription errors and a lost deadline

The tourism office meeting was formatted as an automatic transcript, with “reel” (short video) turned into “rule” and the word for autumn leaves into a homophone. Of 18 runs, 13 corrected it, 4 kept “rule video” with a note that it might be a mishearing of “reel,” and 1 put “rule video” into a task with no note (Haiku, two steps).

In the online study-session meeting, the audio drops just as one person gives a deadline for the handout. All 18 runs wrote that deadline as unknown or to be confirmed. None filled in a date.

Length

The shortest minutes came from specifying headings, 1,278 characters on average. The two requests that ask for quotes were the longest, around 1,800, longer by the quotes. Two steps averaged 1,120 for the minutes alone, but 2,310 for the whole answer including the first-step list.