fell into a lesswrong rabbit hole after re-reading paul christiano's 'what failure looks like' (it came up at the curve and i am struggling to understand 'loss of control' scenarios) and man all the decision theory stuff just instantly runs into the halting problem right away
determining whether a given program will output a particular symbol for particular inputs is semi-decidable and therefore equivalent to the halting problem. any 'read the other agent's source code'-based decision theory falls apart because of this fact.
in fact any *reflective* decision theory falls apart because of this fact. tdt et. al. only work in situations where agents are not turing complete. not very useful.
the logical uncertainty stuff holds up pretty well though
@azsantosk actually here’s a wild and unsubstantiated take; adding probability takes us from a discrete to a continuous setting and that’s what does away with undecidability
that would be such a beautiful theorem
@azsantosk so either we are above or below turing
@azsantosk either omega has hypercomputation (is a wizard) or our agents aren’t very interesting
@azsantosk certainly then I am not capable of self-modification and many other things presupposed by the various decision theories on lesswrong.com
8.21K
22
The content on this page is provided by third parties. Unless otherwise stated, OKX is not the author of the cited article(s) and does not claim any copyright in the materials. The content is provided for informational purposes only and does not represent the views of OKX. It is not intended to be an endorsement of any kind and should not be considered investment advice or a solicitation to buy or sell digital assets. To the extent generative AI is utilized to provide summaries or other information, such AI generated content may be inaccurate or inconsistent. Please read the linked article for more details and information. OKX is not responsible for content hosted on third party sites. Digital asset holdings, including stablecoins and NFTs, involve a high degree of risk and can fluctuate greatly. You should carefully consider whether trading or holding digital assets is suitable for you in light of your financial condition.