isi-nlp / diplomacyamr Goto Github PK

@saramosher888 (with support from @jkkummerfeld ) brought up the following in an unrelated issue thread:
Can we show annotators the board state? This would facilitate the annotation process.

Ulf: The AMR Editor is ready to handle links to board pictures, as demonstrated in workset dip25, for which Denis kindly provided URL links in the AMR workset info file. Denis told me that providing such links is not trivial, so we initially focused on including information about the sender country, the recipient country, and the time (e.g. "Spring 1901").
I support bringing this issue up again at a Diplomacy group meeting where Denis is present.

FYI, I added an example for Scandinavia and Balkans (named-entity type world-region).

https://www.isi.edu/~ulf/amr/lib/amr-dict-diplomacy.html

According to AMR Diplomacy guidelines, we annotate:

and for most ally-01 cases, this is done accordingly, but for a substantial minority, :ARG2 is used (incorrectly), either as a co-ally, or possibly as the enemy. For Diplomacy AMR, :ARG2 should never be used.
Example of a quick AMR Editor command to find such cases:

smosher search ally-01 :ARG2 ...

Here is another issue with the guidelines / checker that is causing consistency problems in annotation. For move-01, we are modifying the unit with the location and the country but leaving :arg0 blank.
Example: "Move the English army in Paris to Brest."

Contrast this with build-01. Until our Github conversation last week, I thought we were handling move-01 and build-01 in the same way, but now I understand that this is not the case. From the beginning, it was made very clear to me that we should be modifying the unit with the :location, like this:
Example: "Build an English army in Paris"

But now the checker is finding an error if I modify the unit with the location. Instead, it wants me to modify build-01 directly, like this:

My concern is that we are not going to have consistency in the annotation if there is this kind of inconsistency in the basic structure of the guidelines. We need to modify the unit in the same way regardless of which concept we happen to be annotating at that particular moment. Otherwise we're setting up a process that's prone to constant errors because the same basic type of information needs to be entered in different places every time.

What's the reason for modifying build-01 with :location directly, instead of adding the location to the unit as we do with move-01?

At first we thought that there was no clear use for the role :instrument. But I have now come across several examples where a player talks about using an army / fleet as support. In these situations, it seems to make sense to use :instrument.

The complication here is that occasionally the power / agent making the move is not specified. In these cases it could make sense to put the unit / army / fleet as ARG1 (agent) of support-01, since otherwise ARG1 would be empty. I lean toward using :instrument for the unit instead, since it's more consistent with the other examples.

Agree / disagree? Is this useful?

1. Used non-Diplomacy AMR concept disband-01. Should probably be remove-01.
2. In dip_train_0009.325, dip_train_0009.326 support-01 has same :ARG0 as :ARG1 (support-01 ENG ENG), which would mean a country supports itself. Could you double-check?

The AMR checker is finding an error for sentences that are annotated with :mode imperative. The error is "unusual agent at :arg0". How should we resolve these? Currently they all have a country name in :arg0, but that's coming back as an error. If we don't want a country name in :arg0, I guess the alternative would be simply not to use :mode imperative. Whenever possible, I've been trying to use propose-01 instead in order to avoid this error, but the sentences below don't fit that model. Thoughts?

Examples:

• "Please lie to Italy." (dip_train_0009.548)
• "I literally just told Italy to go after Turkey first." (dip_train_0012.5)
• "Basically just don't bounce me out of Gas." (dip_train_0022.214)
• "Don't fuck me over tho." (dip_train_0022.328)

English: Build a Russian army in Moscow .

Result-oriented annotation:

(b / build-01 :mode imperative
      :ARG1 (a / army
            :mod (c / country :name (n / name :op1 "Russia"))
            :location (p / province :name (n2 / name :op1 "Moscow"))))

Process-oriented annotation:

(b / build-01 :mode imperative
      :ARG0 (c / country :name (n / name :op1 "Russia"))
      :ARG1 (a / army)
      :location (p / province :name (n2 / name :op1 "Moscow")))

Discussion: Both are reasonable. For consistency, we need to designate a canonical form.

According to AMR Diplomacy guidelines, we annotate:

Specifically the source-location of the transport is annotated as the (original) location of the unit.
This is properly annotated in most annotations, but there are some cases where it's annotated as transport-01 :ARG2 source,
instead of transport-01 :ARG1 unit :location province.
transport-01 :ARG2 should never be used in Diplomacy AMR.
Example of a quick AMR Editor command to find such cases:

smosher search transport-01 :ARG2 ...

Here are some examples of :purpose for discussion of how/whether to use it.

• "Make sure you don't move Munich so that it can take my support." [cutting support or taking support might be one of the most common uses of :purpose.]
• "If Tyr was bound for Trieste anyway, why did you detour through Tyr at all? Why not just move to Trieste last turn?"
• "That means ordering Burgundy to Gascony to cut support."
• "I currently have Tri -Tyrolia. I like the unit there because it sets up an attack on Austria."
• "If you could keep Marseilles open, it will help me to try and take Burgundy next turn."
• "The most important way to get you a build is to take Galicia in the spring. I don't care if you move there or I move there, but we need to make some room around Budapest so that you can build there."
• "I'm getting the fleet out of the way so that I can move on Austria."
• "I'd like the promise of Belgium and Paris to Germany to be a consistent one from both of us, for the purpose of maintaining German confidence in our alliance."

Can / should we use plurals (armies, fleets, units)? Right now the checker is identifying these as errors because it expects to see the singular form of the word. It's not the most frequent issue, but we should probably make a decision.

Examples:

• "Build two armies, which looks like we're really at war and you're going to eject me."
• "She told me that she plans to line the whole northwest coast with fleets."

The AMR Checker is reporting errors whenever there is a country name in :arg0 for remove-01 or transport-01. Is this actually what we want?

For remove-01, the agent of the removal and the country that owns the unit being removed are not always the same. Sometimes the sentence specifies one piece of information but not the other. That means that we lose information if we leave :arg0 blank and only fill in :mod country.

Example:
"Honestly the worst that Germany can do this year is knock me out of the channel."
"If they try to dislodge me instead, I'll retreat to Gascony."
"Dislodging the French army from Bel could result in a disband."
"Italy's fleets may be in position to push you off the island."

For transport-01, I think we do need :arg0 because it seems very common for one country to offer to transport/convoy another country's army. In these cases, :arg0 and :mod would not be the same. And sometimes we don't have both pieces of information, so we don't know whether they are transporting their own army or someone else's.

Example:
"You look poised to Lepanto to Turkey, and it looks like you'll pull it off without the Austrian convoy."
"I can send you past the MAO into West Med."
"Does that mean you'd convoy into Bre? Because I can ask France to support that."
"Germany is convoying Holland to Yorkshire." (we don't know which country owns that unit)
"As long as Italy doesn't try to convoy into Spain from Pie." (we don't know which country owns that unit)

So the question is, do we still want :arg0 to show as an error in these situations?

Let's figure out which AMR roles we are using for this project. Tess and I worked through the list of possible roles and found examples of the kinds of sentences where each might be useful. Some are clearcut, and others are not. Which of these do we want to use in this project, and when?

:year
Players frequently discuss the timing (year, season) of future moves.
Examples:
“It looks like England 's not willing to try for MAO if it means possibly losing the channel . However , they 'll bring the NWG fleet around to try for MAO next year.”
“I think that , in about two years , you and I will both be on about 14 centers , with the remnants of Russia and Austria between us, and we can decide how we want to resolve it .”
“You could just take Denmark this year, and I don’t think England is in a position to retaliate.”

:season
Players frequently discuss the timing (year, season) of future moves.
Examples:
“I’ve now had both England and France suggest to me that I should move to Tyrolia and France will support me in Munich in the fall.”
“What would you think of helping me take Marseilles in two turns?”
“I think we should show him some good faith by supporting him to Brest in Spring. We can decide in Fall whether it makes more sense for you to take it , but I think we want to keep France hungry.”

:time
*No examples of this yet. :year and :season seem more useful.

:beneficiary
The most obvious use of this would be for :support-01, but it’s already built into the frame for this concept. I’m not finding many other clear examples of times when we would use this.
Example:
“Especially if I get two builds this turn , I'll be able to sneak behind the troops in bohemia/galicia and help you break through.”

:cause
:Cause is tricky because there is lots of discussion of why the players are doing various things and what they hope to accomplish, but it’s often very complex and many of the connections are implied. There’s not much discussion of actions that actually (past tense) caused something specific to happen. Most of it is about hypothetical actions that might or might not cause something in the future, and that might fit better with :purpose rather than :cause.
Examples:
“Tell Russia that the last thing in the world you want to see is Austria run him over , and you ’re willing to help keep Russia viable if necessary ( you ’re angling for Russia to disband his northern holdings this turn ).” [because is implied]
“I currently have Tri - Tyrolia . I like the unit there because it sets up an attack on Austria if I ever want to go that route ( build A Ven and go east ).”
“I kind of thought that you would have wanted me in the Channel because it commits me further against England , but I can understand what you ’re saying now about wanting me to hang back .”

:destination
We should probably keep this one because it’s useful for retreat (which has a source but not a destination in its core arguments).
Example:
“Retreat the Italian army from Greece to Albania.”

:duration
This has not come up very often so far.
Example:
“Even if me and France are high-fiving in Bel for a few seasons it 's still mine , and it 's not like Holland has anything better to do while I 'm still allies with England .”

:mode imperative
We definitely need this one for clear orders, but do we also want to use it for more indirect commands/requests?
Example:
“I think you should suggest to England that he gets Sweden and St Petersburg, while you get Denmark back.”

:instrument
No clear examples so far.

:li (list)
No clear examples so far.

:location
:location may be attached to army / unit / fleet
Example: The English army in Liverpool

:mod (modification)
Yes (e.g. relation between unit and country)

:polarity - (negation)
Yes

:purpose
This one is tricky because there is a lot of discussion about players’ reasons for doing various things, but much of the discussion is complex, hypothetical, and involves a lot of half-stated implications. Reasons are not always simple or clearcut. I suspect we are going to get a lot of variation between annotators about how this (:purpose) is marked.
Examples:
“England is leading me to believe it’s part of a play for Belgium.”
“You told me that I need to promise you a set of things in order to take the Channel.”
“Aren't you concerned about England taking Mao? I 'd sooner just have you pile on support into Bre so that Wes can support Mao holding.”
“I think that we should offer France Brest in Spring . That ensures that he is with us.”
“And right now they 're talking to me about supporting a move from Bre to Gas ( the better for the two of us to stab you ).”

:source
So far I can’t think of a time when we would use this, because it is already included in the frames for retreat-01 and transport-01, and we already decided not to use it for move-01.

So far after annotating 300+ messages, I have seen relatively few places where the role :beneficiary is necessary. But here is one example where it might be useful.

Example: "France has always been honest with me, and I am at least sure that they won't betray me to England."

Betray-01 has arg0 (betrayer) and arg1 (betrayed), but no other arguments. So there is no place in this frame to put England. I'm proposing that we use :beneficiary in this situation. Agree / disagree?

Currently the AMR Checker will not allow me to use "and" as :arg1 of demilitarize-01. Often there are two countries agreeing to demilitarize a province, so I need to be able to use "and".

Examples: dip_train_0029.27, dip_validation_0006.8

Do we want to include "tell-01" as a concept for this project? In the data, there is a lot of discussion of who told who what. Players often strategize about what to tell which players in order to manipulate them or convince them to cooperate. We don't use "lie-08" very often because the players usually don't know who is lying, even though they often suspect it. Without "tell-01", much of the content of these messages becomes un-annotatable.

In this example, Germany is uncertain whether England is trying to trick them. Here are two possible annotations, one with "tell-01" and one without.

With "tell-01" the AMR looks like this:

Without "tell-01", the AMR is significantly less informative. Only the part about the proposed alliance is annotatable.

Adding "tell-01" to the concept list would allow us to make annotations much more complete and informative--but it depends on whether this information is useful to the purpose at this stage of the project. Thoughts?

The expanded AMR Checker (or at least the current initial version of it) will signal an error if build-01 is missing its semantic object (:ARG1).
Sara points out that the text often does not include any specific argument for :ARG1, e.g. "I expect to get a build this turn." and wonders whether to put in a "dummy" argument.

I looked at several examples and would suggest to indeed add an unspecified unit in this case, e.g. (build-01 :ARG1 unit)
if it is clear that some military unit is being built.
There are many cases in Diplomacy AMR where we already use unit (which could be an army or fleet).
Adding an :ARG1 will be particularly useful for cases such as two builds (build-01 :ARG1 (unit :quant 2)).

This would also apply to move-01, where unit serves as the head of the country and location (move-01 (unit :mod France :location Brest)).
This would also explicitly certify the sense of moving a military unit.
This contrasts with sentences such as Germany is moving to build an alliance with Austria. or I'm sure that Russia will move on Turkey. with different meanings of move (maybe annotated as attack-01 in the latter case).

I think there is a benefit of identifying cases where the :ARG1 is not a military unit (by marking :ARG1 unit where applicable).

I am also open to changing the argument requirement to a warning (rather than an error) if there are many true exceptions. Let's see.

A little over a month ago, I started supplementary Diplomacy AMR annotation guidelines (https://www.isi.edu/~ulf/amr/lib/amr-dict-diplomacy.html) to complement the general AMR annotation guidelines (https://github.com/amrisi/amr-guidelines/blob/master/amr.md, https://www.isi.edu/~ulf/amr/lib/amr-dict.html) and announced at our weekly meetings, our slack channel and the AMR Diplomacy Google doc.

Today, Tess made multiple references to Sara's list of AMR guidelines, which seem to at least partially overlap.

Could you let us know how you propose the two relate: Is Sara's list meant to be something of a scratch list with ideas that might go into the existing guidelines, or a replacement, or something complementary?

The resource I created is of course work in progress. Feel free to suggest additions, clarification, or changes. I have not had any feedback so far.

People who play Diplomacy: What does "bounce" mean?

Example:
Italy to Austria: "So, do I just convoy my army to Tunis? Also, do you want me to bounce you to Trieste?"
Austria to Italy: "Sure, Trieste bounce is a good idea. And standard Lepanto with the convoy."

I've seen this several times now from different players, and I'm still not sure from context what they are talking about. Maybe it's convoy / transport?

I noticed a number of "unit" entities located at seas or coasts.
Wouldn't those all be fleets?
We need "unit" if we are not sure if it's an army or a fleet, but a sea or coast location should mean it's a fleet, even if that word does not occur explicitly.
For the time being it's now flagged by the AMR Checker, but I'm open to new insights.

I propose to collect principles to build consensus for Diplomacy AMR annotations, including why they are important.
I'll start with a few. Additions, feedback, discussion welcome.

I agree with Sara that for Diplomacy, legitimate world-region entities include:

• Scandinavia
• Balkans
• Britain
• Iberia

I updated the AMR Checker in that respect.

TL;DR where there are two arg structure options for AMR and we expect sentence meaning to be equivalent in our data, is there a default principle that would work, such as taking the option that distinguishes fewer ARGs in the top frame (e.g. for PREVENT and ALLY below)? Or since we have a limited number of frames, should we just resolve frame by frame?

How to get consistent annotation for frames which allow multiple arg structure options that don't make a significant difference for Diplomacy data?
For example:

1. PREVENT
(1) X prevented Y’s attack on Z
(2) X prevented Y from attacking Z
(3) X prevented the attack on Z [where Y is known to be the attacker from a previous sentence]

prevent-01 has

• ARG1: Theme (action or object being prevented)   theme
• ARG2: secondary predication or action   theme

So assuming these are all to have the same AMR, do we want to default to
p / prevent-01
:ARG0 X
:ARG1 Y
:ARG2 attack (:ARG0 Y, :ARG1 Z)

or

p / prevent-01
:ARG0 X
:ARG1 attack (:ARG0 Y, :ARG1 Z)

2. ALLY
ally-01 has the option to distinguish
ARG0: agent, causer
ARG1: entity united
ARG2: entity united with ( co-agent)
(4) X is forming an alliance with Y
(5) X and Y are forming an alliance

a / ally-01
:ARG1 X
:ARG2 Y

a / ally-01
:ARG1 a2 / and (:op1 X, :op2 Y)

PROPOSAL: would it work to pick the option that has fewer distinct ARGs and use it consistently? i.e. take the second option above for both prevent and ally ? This would be consistent with what we've done so far for "move" and "retreat", plus the result-oriented annotation for "build".

ALTERNATIVE: just have a frame list where we pick one by one the version that seems most appropriate for our purposes.

The expanded AMR checker is also signaling an error if there is a country in :ARG0 for build-01 or move-01. For most of the concepts we are using, ARG0 is the agent/country. From the annotator's perspective, it seems more confusing than necessary to make ARG0 different for these two concepts than it is for all the others. In the argument structures listed in the main AMR dictionary, build-01 has "builder" as :ARG0, and move-01 has "mover" as :ARG0. So is there a reason why are are not supposed to use slots that already exist for this purpose?

I should mention that the frame key that I have been using as my regular reference (which is based on the frame pop-up that shows in AMR Editor) has :ARG0 as the agent / mover / builder. It seems that we did not catch the discrepancy between that and the Diplomacy-specific dictionary that Ulf is maintaining. So up to this point, all of the 1,000+ sentences that have been annotated have the country in :ARG0 for build-01 and move-01. In other words, the annotation is consistent--but it's consistent about filling in :ARG0 instead of leaving it blank. It is certainly possible to go back and change them all, but that's expensive in terms of time if they all need to be changed one by one.

So is it important to leave :ARG0 blank? If it is important, is it possible to write some code that can change all of them automatically?

In the data, "trust" is an important concept which is not possible to annotate under the current guidelines. Is it useful at this stage of the project to include "trust-01" as an annotatable concept, or should we continue leaving it out?

Here are several examples of messages that not possible to annotate completely without "trust-01".

Previously I mentioned an issue in AMR Editor where the variables assigned to country names can get scrambled if the first mention of the country name is deleted and re-added. This is coming up repeatedly whenever we need to make corrections.

In this example, England used to be c2. The first mention of England was in :arg0 of move-01. When I deleted :arg0, that removed the first mention of England where the country name was spelled out. When I re-added it as u :mod ENG, England was assigned a new variable (c5). But all the other mentions of c2 stayed the same. So the different mentions of England are no longer connected to one another. Every time I remove :arg0, that leads to a cascade of manual changes throughout the rest of the AMR in order to make sure that the variables are still consistent. In this process it's very easy to miss something or make a mistake.

Is there a better way to do this?

This example is dip_train_0004.188.

We're nearly out of new worksets to annotate, and we are almost done cleaning up errors from the finished ones. We'll be ready to start new work in a day or two. Can we get access to some more worksets?
Thanks!

Let's come up with a standard way of annotating time / year / season. In the below example, Tess put :time "last season", and I put :season "spring 1902". What is the preferred way?

In yesterday's meeting, it sounded like the best way would be to include both year and season. @uhermjakob, what is the format that you would like us to use?

In Diplomacy, holding a unit (of some power at some location) means to hold in place (as opposed to move it or have provide support).
So I expected an annotation of type hold-03 :ARG1 unit

But I see many instances with an hold-03 :ARG0 ... or hold-03 :ARG1 province, which differs from the Diplomacy command HLD (in DAIDE).

Merging consecutive messages from the same user could make annotation easier when a player spreads the same thought over more than one message, or when they immediately send a second message to finish or clarify their previous statement. Is it possible to merge consecutive messages from the same user? If so, it would be a good idea to include some kind of indicator to the annotator that there is a message break, in case it matters to the meaning.

I would not say that this is an urgent issue, but if it is easy to do it might be worthwhile.

FYI, I made a number of adjustments to the AMR Checker to reflect some of the special conventions of Diplomacy AMR.
I now recommend that annotators regularly use it to perform a sanity-check on the AMRs they annotated.

Errors that the AMR Checker catches include:

• non-existing frames (such as have-01)
• non-existing core roles for a given frame
• :op1 instead of :ARG1
• ill-formed date-entity (e.g. for "spring 1902")
• ill-formed names
• strings ("c2") instead of variables (c2)
• duplicate roles
• cycles

How to use the AMR Checker:

• In the AMR Editor, click the blue check button (towards the bottom left).
• Select annotator and workset (defaults provided)
  • To check all Diplomacy worksets for an annotators, specify workset dip-all
• Unselect Check sentence coverage
• Click on blue Check AMRs button
• Results will indicate errors (probably an error) and warnings (maybe an error)
• Results include blue Edit buttons to directly proceed to an AMR with problems.
  

Some of the adjustments of the AMR Checker for Diplomacy include:

• Better supports Diplomacy orders
• Connects Diplomacy names and IDs of powers and provinces (e.g. AUS = Austria)
  • used for alignment
  • allows "Berlin" and "Sweden" to be of NE type "province", rather than the standard "city" and "country"
• Support for Diplomacy's unspecified military unit unit (army or fleet)
• Added Diplomacy-specific alignment-support file
  • licences certain additional function words
  • connects betray-01 also to "stab", "traitor" (incl. morph. variants)

With the modifications, many false positive errors and warnings have been eliminated.
Very high precision for errors, high precision for warnings.

To do (Ulf):
Manually inspect annotated AMRs to add more AMR Checker tests to increase recall of errors and warnings.

Can we allow country names in unit :location ? The issue is that occasionally a player will refer to units that are located in a country/region rather than a specific province. (This doesn't happen very often).

Example 1: "I may just pull back from Turkey to cover up the north myself." (dip_train_0023.555)
In this example, AMR Checker rejects Turkey as a location because it is not a province name. I can't tell from the map whether the player means SMY or ARM.

Example 2: "I'm going to try to take Norway from England if that helps any." (dip_train_0029.31)
In this example, AMR Checker rejects England as a location because it is not a province name. According to the map, the player could have been referring to units in LVP, EDI, or LON.

This issue is probably not our highest priority to resolve, since I currently only have two examples of this problem. But I need to decide what to do with these two.

I've been seeing occasional messages about losing or gaining supply centers. Currently I don't think we have a way to annotate this. Is this something we want to pay attention to, or should we continue leaving it out?

Examples:

• "The convoy makes me a bit nervous because if France goes to IRI, they'll get a home supply center in the fall."
• "If I lose the channel, it won't be for very long, but if I lose a supply center it will be very annoying."
• "I'm not so sure you should read that seizing of Trieste as an act of war; Italy did help Austria take two supply centers this turn. Austria might have given it willingly."
• "Austria can't build. All of its home supply centers are occupied."

English: France 's move into Belgium

Annotation with unit

(m / move-01
      :ARG1 (u / unit
            :mod (c / country :name (n / name :op1 "France")))
      :ARG2 (p / province :name (n2 / name :op1 "Belgium")))

Annotation with :ARG0

(m / move-01
      :ARG0 (c / country :name (n / name :op1 "France"))
      :ARG2 (p / province :name (n2 / name :op1 "Belgium")))

Discussion: Both are reasonable. For consistency, we need to designate a canonical form.

In the data, we're seeing a lot of examples where it would be useful to annotate that a player thought some action or outcome would be possible or impossible. "Possible" is different from "expect" because the player is not necessarily stating what they think will happen, but only what could happen. Currently we don't have a good way to annotate this. Adding "possible-01" to our concept list would help us deal with discussions of hypothetical situations.

Examples:
It would have been useful to have "possible-01" as an option for this one.

Here is an example where it would have been more accurate to use "possible-01" instead of "expect-01".

Here is another one where I used "possible-01", and it made the annotation clearer than any other option I could come up with.

Let's decide on a standard way to mark "bounce". This one is a little tricky because we don't always have the same pieces of information available. Depending on how the word is used, we might be missing the agent, the unit, or the country that owns the unit. Or there might be more than one instrument / unit / fleet without knowing which country they belong to.

Examples:

• "You may want to use my fleet to bounce with you at Con to keep the Russian fleet out."
• "Arm and Smy bounce in Ank, Tyr and Gal bounce in Vienna."
• "Would you mind asking Austria again for a bounce in Galicia?"
• "Can you tell Germany to self-bounce in Ruhr?"
• "I think your best bet may be Picardy-Brest (expect a bounce).
• "I just realized that I really have to hope for a bounce in Edi or London."

Our options are:

• move-01 (arg0 is agent, arg1 is the unit, arg2 is the destination). This is not a great option because there is usually more than one unit involved and we don't usually know a destination.
• prevent-01 (arg0 is the agent, arg1 would be move-01, and arg3 is the instrument / unit).
• remove-01 (arg0 is the agent, arg1 would be the unit, and arg2 would be the location it was removed from).

We have been keeping a list of sentences that mention "bounce" so that we can go back and standardize them once we've made a firm decision about the best way.

Sometimes :arg1 is left empty because the sentence doesn't specify what is being supported. This usually comes up in sentences that are about cutting support. This is showing up as an error in the checker. How would you like us to annotate sentences discuss "support" without specifying what kind of action is being supported or or what support is being cut?

Examples:

• "Can you please order MUN to BOH to cut support?"
• "You're not actually moving to PIC, you're just cutting support."
• "I'm moving to HEL to cut support."
• "Was that the reason you gave for not supporting?"
• "If Italy and I "agree" to a plan where he does some supporting from Mao, and he says nothing to you to contradict that plan, would you take that as enough proof that IRI is safe?"
• "He wanted me to cut support from BOH, and I refused."

Diplomacy players, can you tell me what "tap" means?

Examples:
"Right now I've got my fleet in Con tapping Black, so it'll probably stay full. Get the fleet to where it can help with France."
"Use Tyr to tap Bohemia, Vienna supports Budapest to Galicia, and Romania taps Ukraine."

Also in connection with that first example, I'm guessing from context that Germany is black, but I'm not sure. Is there any way to know?

The annotations contain a few cases with retreat-01 :ARG1 country, but in Diplomacy, technically, unit retreat, not countries.
These cases should be expressed as a unit (army or fleet) from that country retreating: retreat-01 :ARG1 UNIT :mod country
Example of a quick AMR Editor command to find such cases:

smosher search retreat-01 :ARG1 country

Sometimes hold-03 doesn't have any :arg1 specified, but the checker turns up an error if we leave :arg1 blank. It's possible that if we had access to images of the board state, we might be able to figure it out. But I can't be sure of that. If we can't figure out what is being held, is it better to leave :arg1 blank or just leave the hold out of the annotation altogether? (The vast majority of the time we can tell what is being held--but here are 3 examples where we couldn't figure it out.)

• "If Italy goes for Spa, the choice is between a hold or an attack from Pie. I would say a hold makes more sense for Italy if you've decided you're going for Bur."
• "I'll support hold from NWG just in case."
• "Actually the support hold is unnecessary."

Are there more worksets to annotate? Right now I'm correcting some errors in the ones I've already done, but will be ready for new ones soon.

When player 1 proposes something and then player 2 accepts or rejects that proposal in a later message, is it necessary to re-annotate all the content of the original proposal?

Unless there are objections, I propose that we don't re-annotate the content of the original proposal when we annotate "accept" or "reject" in a subsequent message. 1) It takes extra time to annotate the content of the proposal twice. 2) It's not always immediately clear exactly which aspect of the proposal the other player is agreeing to.

In the example below, I spent several minutes re-annotating the content of the proposal under "agree-01", only to realize in the next message that Germany was not agreeing to all of this.

Example:
Italy to Germany: "I think England will want to coax me to attack you with him after France falls, but I'd much rather work with you against England. But first thing first--let's get rid of France."
Germany to Italy: "Agreed."
Germany to Italy: "(On the France part.)"

Is there a strong reason to repeat the content of a proposal when annotating a subsequent acceptance or rejection of the proposal? If not, I vote for the quicker option.