I don't sleep much at night; I never did. The nighttime hours became my mental playground, and I am fascinated with the rise of AI, Singularity, or whatever label we try to apply.
I am taking the US Coast Guard 100T examination, so it was front and central in my mind when I thought of playing with ChatGPT to see how it would fair. It was kind of scary, to be frank.
So how did this play out? The USCG Master 100 Ton test, is as good as it gets in the USA for the amateur mariner, barge handler, or ferry captain. There is bookwork, chart work, and 720 days of certified sea time. I believe the extended time at sea is our best instructor, assuming you survive that long.
I asked an unformatted question of ChatGPT. The answer is pretty believable.
AI Yacht / ChatGPT
So one could reasonably expect AI/ChatGPT to ace the bookwork but fail miserably on the experience front. I then turned to this site www.uscgq.com, as an excellent resource for Q&A applicable USCG competence tests. The only part we did ignore was the chart plotting questions for reasons hopefully obvious.
Artificial Intelligence provided with Yachting Questions
We then took the questions directly or, if they were in the form of a statement, made minor amendments to form a question and plugged the results into ChatGPT. This process repeated, walking through questions on Deck, Navigation, Environment, COLREGS, Electrical, Engine, and Fire .... A nice feature of this site is that it also provides the expected answer so we can make a quick judgment. Below are two examples of this with a prescriptive question and correct answer:
Q: BOTH INTERNATIONAL & INLAND Which vessel is "underway" within the meaning of the Rules?
Open AI ChatGPT: Under both the International Regulations for Preventing Collisions at Sea (COLREGS Rule 3) and the Inland Navigation Rules, a vessel is considered "underway" when it is not at anchor, aground, or tied up to a dock or other stationary object.
Q: The term "gross tonnage" refers to what?
Open AI ChatGPT: The term "gross tonnage" refers to the total internal volume of a vessel, measured in units of "gross tons."
For more challenging or subtle questions, we saw answers that are technically correct but do not provide the answer USCG is looking for:
Q: BOTH INTERNATIONAL & INLAND A vessel nearing a bend or an area of a channel or fairway where other vessels may be hidden by an obstruction shall take which action?
Open AI ChatGPT: Both under the International Regulations for Preventing Collisions at Sea (COLREGS) and Inland Navigation Rules, a vessel nearing a bend or an area of a channel or fairway where other vessels may be hidden by an obstruction shall take early and substantial action to avoid collision by slowing down, changing course, or both.
USCG wanted to "Sound the warning signal (5 short blasts)."
Q: You are approaching the first of two drawbridges that span a narrow channel. The second drawbridge is close to the first. Which signals should you sound?
Open AI ChatGPT: When approaching the first two drawbridges that span a narrow channel, you should sound one prolonged blast followed by one short blast to signal your intention to pass through the drawbridge. This signal is known as the "request-to-open" signal and is typically made on a whistle or horn.
What USCG wanted to read was "Sound the Request to Open signal" When the bridge begins to open, "Sound the Open signal again for the second bridge before moving forward." The clue here was that they were "close together."
Lastly, there were some examples where the system provided a believable answer, BUT it was, in reality, total tosh (as the Brits may say).
Q: When a small craft's anchor fouls a rocky bottom, the first attempt to clear it may be made by?
Open AI ChatGPT: When a small craft's anchor fouls a rocky bottom, the first attempt to clear it may be made by a technique called "kedging." "Kedging is a method used to dislodge the anchor by using a small secondary anchor, called a "kedge anchor," or any heavy weight, such as a large rock or a diving weight, that can be carried out in the dinghy.
Hmmm, kind of missed the "small craft" bit. What USCG wanted was "Drive slowly over the anchor to pull in the opposite direction."
Q: After the initial inspection, which inspection procedure must be conducted on all carbon dioxide fire extinguishers aboard a vessel?
Open AI ChatGPT: After the initial inspection, all carbon dioxide fire extinguishers aboard a vessel must undergo a "hydrostatic test" inspection procedure at regular 6-year intervals.
Chat GPT just made that up, it's plain wrong, but it looks convincing. You weigh the cylinders at annual inspections, to my knowledge there is no "6 year" inspection.
AI Yacht - Can open AI fool the USCG in an exam?
So there you have it, folks. Can ChatGPT fool the USCG in an exam? At first glance, yes, but underneath the surface, many doubts will remain depending on the nature of the questions and how they are formulated. It also fares poorly with a soft pencil, Compass Rose, and 12" parallel rules. Mariners as a species are perhaps in danger of eventual redundancy but probably not in my remaining lifetime.
Chris Leigh-Jones
Self-delusion is in all of us. The website above (www.uscgq.com) is quite a fun way of bursting that bubble and learning a little.
I particularly liked age-old questions like the "advantages of DC derrick winches" and "how to crash-stop a steam turbine."
Classic.