• 3 Posts
  • 12 Comments
Joined 1 year ago
cake
Cake day: July 19th, 2023

help-circle
  • Even better, we can say that it’s the actual hard prompt: this is real text written by real OpenAI employees. GPTs are well-known to easily quote verbatim from their context, and OpenAI trains theirs to do it by teaching them to break down word problems into pieces which are manipulated and regurgitated. This is clownshoes prompt engineering done by manager-first principles like “not knowing what we want” and “being able to quickly change the behavior of our products with millions of customers in unpredictable ways”.


  • That’s the standard response from last decade. However, we now have a theory of soft prompting: start with a textual prompt, embed it, and then optimize the embedding with a round of fine-tuning. It would be obvious if OpenAI were using this technique, because we would only recover similar texts instead of verbatim texts when leaking the prompt (unless at zero temperature, perhaps.) This is a good example of how OpenAI’s offerings are behind the state of the art.


  • Not with this framing. By adopting the first- and second-person pronouns immediately, the simulation is collapsed into a simple Turing-test scenario, and the computer’s only personality objective (in terms of what was optimized during RLHF) is to excel at that Turing test. The given personalities are all roles performed by a single underlying actor.

    As the saying goes, the best evidence for the shape-rotator/wordcel dichotomy is that techbros are terrible at words.

    NSFW

    The way to fix this is to embed the entire conversation into the simulation with third-person framing, as if it were a story, log, or transcript. This means that a personality would be simulated not by an actor in a Turing test, but directly by the token-predictor. In terms of narrative, it means strictly defining and enforcing a fourth wall. We can see elements of this in fine-tuning of many GPTs for RAG or conversation, but such fine-tuning only defines formatted acting rather than personality simulation.








  • This is some of the most corporate-brained reasoning I’ve ever seen. To recap:

    • NYC elects a cop as mayor
    • Cop-mayor decrees that NYC will be great again, because of businesses
    • Cops and other oinkers get extra cash even though they aren’t business
    • Commercial real estate is still cratering and cops can’t find anybody to stop/frisk/arrest/blame for it
    • Folks over in New Jersey are giggling at the cop-mayor, something must be done
    • NYC invites folks to become small-business owners, landlords, realtors, etc.
    • Cop-mayor doesn’t understand how to fund it (whaddaya mean, I can’t hire cops to give accounting advice!?)
    • Cop-mayor’s CTO (yes, the city has corporate officers) suggests a fancy chatbot instead of hiring people

    It’s a fucking pattern, ain’t it.