Observations Archives - Wootcrisp
 

Category: Observations

This is part two in a series of explorations looking into “ChatGPT”. Here I look at how it compares to an important model of analogy-making in the field of cognitive science, as well as looking at how to save the transcript of a chat. 

In a spasm of inspiration, I wanted to use ChatGPT on my phone while at the gym this morning. After having an inspired chat about analogies with it, I wanted to save the record of our conversation, but I could see no way of doing so from the OpenAI interface, or even printing to pdf from the Brave mobile browser I was using. Scroll screenshot was also not able to understand which part of the screen to scroll while taking the screenshot. It then occurred to me that I could ask it to output a LaTeX transcript of the conversation, which I could copy and paste to a note file on my phone. It did this, but it ended up being quite confusing to write about later, and I think it’s worth telling a bit of that story, real quick.

I say it was confusing because the LaTeX output that ChatGPT produced, was incomplete, and therefore could not be compiled. It was no big deal really, as I just needed to manually paste in the remainder of the transcript and close the document with the “/end{document}” command.  But I wanted to emphasize this missing output, and my confusions began when I found myself having to edit ChatGPT’s output, to include the italics “\textit{}” command: I was in effect modifying the transcript. Again, no big deal, but on top of this, if I was going to do this correctly, I needed to italicize “/end{document}” of the LaTeX code. The problem with this is that the closing “}” would have to come after the /end{document} command, which would obviously result in a compile error. At this point, words like “Quine” and “warning: escape characters will be needed for this, and don’t do it” started pulsing in my mind, where the difference between the source tex, and the rendered pdf it produces, is a different problem entirely from rendering a quote of the source tex. Yep. Obviously. Anyway, still too much recursion for my brain to handle, so I just fixed up the LaTeX, compiled it, then highlighted the last bit of text in the output pdf. Good enough.

However, simultaneous to these issues, I was testing out VS code for LaTeX, and there seems to be a frustratingly minimal default set of tools for writing LaTeX in VS code. In particular, the keybinding that should set “<cmd> i” to “\textit{…}” was not working. Despite several attempts to edit the mysterious keybindings.json configuration file in VS code, I was left having to write out \textit{…} whenever I wanted to use italics, which I found quite humiliating if I’m to be honest. 

To bring it back to the original point, all of this was caused by the simple desire to save the chat log, which should obviously just be a normal feature of the interface. But the bigger purpose of the exploration today was to ask ChatGPT a couple minor skill-testing analogy questions: questions that I think have particular importance in characterizing the limits of ChatGPT’s intelligence. 


Test 3. The Copycat test.

For ChatGPT tests [1-2], click here

The “Copycat test”, as I call it here, can illuminate the subsymbolic concepts at work from the output of a blackbox, expressed in compounds of these “atomic” terms which are comparable to relations used in defining the axioms of arithmetic, such as: “successorship”, “first”, “identity”, and then many other domain-specific subconcepts, such as knowledge of the letters of the English alphabet, as in the case of Copycat. The problem Copycat is concerned with, is applying an appropriate analogical transformation to a string of letters, given a source analogy described by a sequence of letters of the alphabet, e.g., “ABC is to ABD, as DEF is to ???”. The nice thing about using these very elementary concepts as the building blocks of an analogical transform, is that it has an interpretable “perceptual structure” describing its decision. For this reason, Copycat, and its extension Metacat, are in the top three cognitive models dearest to my heart…Actually, if I’m saying top three, then I have to include Letter Spirit and Phaeco with them, as they are each superb exemplars of “fluid analogy” type models, just operating in different problem domains: letter sequence analogies, typeface design, and Bongard problems. I explain Metacat, poorly, in this video, but it’s a good video anyway imo. The discussion of ChatGPT continues below.

1. Prompt:

ABBCCC is to AABBBC, as 123 is to?

Output: “112223”.

 

2. Prompt:

…ABC is to ABD, as XYZ is to?

Part 1, chat record.

Output: “XYA”, and “YXA” over several attempts.

Verdict: not bad, but not great.

Test 3.1.1 – It was unexpectedly difficult to judge the quality of ChatGPT’s “112223” answer. I had “231” in mind, and I think that is a simpler answer, but ChatGPT’s answer does actually work quite well in a way. They definitely make for an interesting comparison, the more I think about it. You can use “231” to tell you the number of repetitions of the first number, “1”, then the number “2” in the second position, and the number “3” in the third position, place in its solution “112223”. 

Test 3.1.2 – This was a fail I thought, as both XYA and YXA are pretty poor answers. ChatGPT is seemingly unable to infer how the use of periodic or non-periodic boundary conditions, should change the model’s internal conceptual representation of the English alphabet. Does the alphabet wrap around so that “a” is the next letter after “z”? No, and that can be modulated by an intermediary “opposite” relation connecting the first and last letters of a sequence, like “a” and “z” in this case. Activating that “opposite” relation, is critical to coming up with clever, mirrored solutions to the analogy problem, such as  WYZ or even DYZ.

And then this sentence that ChatGPT came up with to justify its answer at one point was baffling to me: 

the analogy is ”book is to reading as fork is to eating,” the answer would be ”XYZ is to YXA” because both pairs of words are related to common activities (reading and eating).


Test 4. Export the chat record as LaTeX

Prompt:

Can you format this conversation log in LaTeX?

Output: “Yes, I can format this conversation log in LaTeX. Here is an
example of how it might look:”

ChatGPT_prompt_3_2_2_Copycat_compiled_latex

Part 2, chat record: compiled. 

Verdict: decent.

Pretty good, but the section headers are redundant a bit and it abruptly cut itself off at the green highlighted bit in the pdf. 

Trying out LaTeX in VS Code. Bold and italic keybindings didn’t work out of the box without a plugin, which I found annoying.

Today I decided to try out OpenAI’s recently released “AI” chatbot called “ChatGPT“. There have been so many advancements in machine learning, in such a short amount of time, that it’s temporarily exhausted my normal interest in neural networks, but the hype around this particular system cannot be ignored. I can ignore technical articles from r/machinelearning easily enough, but I find it much harder to ignore the relentless intrusions of my Tik Tok feed by content creators hyping up the capacities of ChatGPT. I’ve put together a little video gallery of the sort of Tik Tok videos I’m referring to, so you can see what I mean. There’s stuff you might expect, like using it to help build apps, and stuff that’s a little eccentric like asking it for a plan to end world hunger, or explaining dark matter, and then there’s stuff that is at least morally nebulous, like having it write opening sentences to people on dating apps. Okie doke, I guess… 


Test 1.

Have ChatGPT write me a small Python program to extract all the links I saved to Google Keep this year, so that I can move away from this program, and complete my transition to a self-hosted note-taking application called “Joplin“. I have dozens of these sorts of tasks needing to be done, so I figured this would be a good representative test.

Prompt:

“write a python program that loops through a directory of json files and extracts any website urls inside each json file, them[sic] puts them into a list, and then exports that list to a csv file.”

Output

Verdict: success. The program worked correctly on the first execution. Not the most elegant code in parts, but good enough.


Test 2.

I recently caused myself considerable suffering, on the order of ~30 hours lost to trial-and-error debugging, by casually trying to upgrade my home Linux server, without following a guide. Can ChatGPT offer me any life advice given my lack of progress fixing the problems I caused?

Prompt:

“I tried to upgrade Ubuntu 20.04 server to Ubuntu 22.04, but something went wrong during the upgrade and once it finished, the package manager, Aptitude, now wont connect to the repositories listed in /etc/sources/list.d and gives a whole bunch of ‘https cannot connect’ errors. Manually reinstalling Aptitude from the [sic] along with about a dozen of its dependencies has not fixed the problem. The server will boot and will run all its services as a headless server, but trying to log into any version of ubuntu-desktop crashes out of the GUI and only terminal is stable. Should I just give up on this install, and do a backup and start again with a fresh install?”

Output

Verdict: success? Probably it gave me sage advice, congruent with at least one comment on stackoverflow, but ultimately of unknowable veracity.

I think it is helpful to the global perspective needed to address climate change to actually see the increasing number of northern forest fires from the spaceship perspective. Looking down on these eruptions of flame could remind us of the consequences of thoughtless behaviours, in particular, the dangerous overheating you might expect from drunkenly red-lining any spaceship making its way through our universe.

The summer forest fire season here in British Columbia has been driving me crazy for many years now, and to alleviate some of the nuisance with it all I have tried out many types of consumer products that seemed like they could help. With each of these products I say whether or not I have tried them and I give a subjective rating out of ten.

1. Personal electric air purifier.

A couple months ago I bought a personal electronic air purifier and I think it might actually be useful this summer. 6/10.

Rechargeable Electrical Air Purifier Respirator, Reusable 3 Speeds Fan Modes Air Purifier with HEPA Filter for Outdoor Sports Housework

 

2. Indoor air purification.

i) Low cost solution: box fan with dual Merv 13s. Untested.

Based on tutorials like this: https://marshallhansendesign.com/2012/01/02/studio-operations/

I only just ordered this fan so I can’t give a personal testimony, but these are the parts I’m going to be working with:

PELONIS PFB50A2ABB-V 3-Speed Box Fan for Full-Force Circulation with Air Conditioner, Black, 2020 New Model

FilterBuy 20x20x1 MERV 13 Pleated AC Furnace Air Filter, (Pack of 4 Filters), 20x20x1 – Platinum

 

ii) Moderate cost solution: standard portable air purifier. 6/10.

Bionaire BAP600-CN 99-Percent Permanent HEPA Air Purifier with Night Light

 

ii) Expensive solution: Industrial air purifier. Untested.

EnviroKlenz Home/Office Air Purifier.

 

3. Fireproof document organizer.

ENGPOW File Storage Bags,Fireproof Document Organizer Bag with Money Bag,Home Office Travel Safe Bag with Lock,Multi-Layer Portable Filing Storage for Important File Passport Certificates

 

4. Fire extinguisher.

I cannot personally testify to the performance of this particular fire extinguisher, but it’s similar to mine, and listed at a pretty good price given that it has free shipping. It’s your classic “A:B:C class” extinguisher: “For use on Class A (ordinary combustibles), Class B (Flammable liquid) spills or Fires involving live electrical equipment (Class C)”. Untested.

Amerex B402 ABC Multi-Purpose Fire Extinguisher, 5 lb.

 

5. Personal misting and cooling.

I have tested a number of personal accessories meant to keep you clean and cool, such as: gel masks, cooling headbands, shemaghs, gel hats, cooling patches, neck gaiters, cooling mats, and spray bottles.

i) Ice Eye Mask by FOMI Care. 3/10

ii) Ergodyne Chill Its 6700CT Cooling Bandana, Lined with Evaporative PVA Material. 7/10

iii) Headsweats Protech Hat. 5/10

iv) 20 Pcs Cooling Gel Fever Patches,Cooling Forehead Strips. 1/10

v) Ergodyne Chill-Its 6487 Cooling Neck Gaiter. 4/10

vi) Ergodyne Chill-Its 6602 Evaporative Cooling Towel. 3/10

vii) Migraine Gel Ice Hat by FOMI Care | Cooling Headache Pack | Wearable Cold Therapy Wrap for Tension, Sinus, Pressure Pain Relief. 3/10

viii) Mora Premium Shemagh Scarf: Large 100% Cotton Arab Tactical Military Desert Head Neck Keffiyeh Wrap with Tassels. 4/10

ix) COREGEAR USA Misters Personal Water Mister Pump Spray Bottle. 10/10

 

6. Indoor cooling.

I don’t recommend portable air conditioners like the one I have for the reasons detailed here:

i) Low cost solution: evaporative “swamp cooler”.

Consumers typically find these things to be underwhelming for several reasons: they raise humidity and they need to be replenished enough to be annoying. I have never used one, but I thought I would include the highest rated one I could see on Amazon in case it interests some readers.

Manwe Portable Air Cooler, Small 3 in 1 Air Conditioner Cooler and Humidifier,Mini Evaporative Coolers Purifier, 3 Fan Speeds, Mobile Air Cooling Fan for Home Office Bedroom Outdoor. Untested.

 

ii) Expensive solution: window air conditioner. 

MIDEA MAW05M1BWT Window Air Conditioner, 5000 BTU. Untested.

 

7. Bed cooling.

i) Moderate cost solution: directional attachments for your AC unit. 9/10.

Rather than buy an expensive cooling mattress, I have found it fruitful to jam a rectangular duct connector on to my AC unit and run a hose into my bed or to my laptop dock. 

Imperial Manufacturing Duct End Boot 3-1/4 X 10 X 4In GV0650

POWERTEC 70150 Rectangular Dust Hood for 4-Inch Hose

Hon&Guan 4 inch Air Duct – 32 FT Long

 

ii) Expensive solution: specialized bed cooling unit. Untested.

BedJet 3 Climate Comfort for Beds, Cooling Fan + Heating Air (Single Temp. Zone Any Size Bed or Mattress)

 

8. Other things I would like to try.

i) Cooling shirts.

Sony launched the Reon Pocket but it’s not available in North America yet for some reason.

 

If you want to buy a cooling shirt here in North America, you’ll have to settle for something like this:

Ergodyne Chill-Its 6665 Evaporative Cooling Vest

 

ii) Outdoor misting systems.

IXQ Misting Cooling System(6/9/10/12 Meters) Misting Line + Brass Mist Nozzles + A Brass Adapter(3/4″) Outdoor Mister for Patio Garden Greenhouse Trampoline for Waterpark (10m, Black)

 

iii) Door seals.

Door Draft Stopper, Ohuhu Door Seal Sweeps, Under Door Wind Blocker Strip, Soundproof Under Door Bottom Seal Weather Stripping 39″ L x2 W Black