Comments for bit-player http://bit-player.org An amateur's outlook on computation and mathematics Thu, 13 Jun 2024 01:29:55 +0000 hourly 1 http://wordpress.org/?v=3.4.2 Comment on World3, the public beta by Keith Hayes http://bit-player.org/2012/world3-the-public-beta/comment-page-1#comment-486148 Keith Hayes Thu, 13 Jun 2024 01:29:55 +0000 http://bit-player.org/?p=1172#comment-486148 Brian, It is a delight to find you have a website and have implemented the Limits to growth equations. You implemented World 3 by hardwiring the equations. I am doing something a bit different. My goal is to run the original card deck on a Dynamo Compiler simulator I am writing. At 1300 lines of JavaScript I can load and run the short card deck from the Dynamo Users Manual. I have been working on the project for about a month. It sorts variables and implements the core equations using function pointers and arrays as arguments. What you have done will be a great help. I have been interested in resource depletion issues for a long time, and have been a 'believer' in the limits to growth model since it was published. Whatever that means, I don't have to explain because I have watched your lecture. https://www.youtube.com/watch?v=47-deZ8uT-w Your video is on my You-Tube channel and has had 2.2k views so far. I put it up on my You-Tube channel a couple of years ago. When I posted your video it was from my interest in collapse, environmentalism and such. I forgot how I found it and I did not Google who you were until today. My interest in building the Dynamo simulated renewed my interest, and I re-watched the video. I have Forester's Book and The Dynamo Users Manual. An original copy of Limits To Growth is supposed to be in the mail on the way to me. The second edition is of little use to the project. I would greatly appreciate getting my paws on a copy of the original LTG card deck if you have a copy. My goal is to have a page on my own website where the original card deck can be run, and where a user can load their own files. I want users to be able to write their own models. Thank You, Keith Hayes Brian,

It is a delight to find you have a website and have implemented the Limits to growth equations. You implemented World 3 by hardwiring the equations. I am doing something a bit different. My goal is to run the original card deck on a Dynamo Compiler simulator I am writing. At 1300 lines of JavaScript I can load and run the short card deck from the Dynamo Users Manual. I have been working on the project for about a month. It sorts variables and implements the core equations using function pointers and arrays as arguments. What you have done will be a great help.

I have been interested in resource depletion issues for a long time, and have been a ‘believer’ in the limits to growth model since it was published. Whatever that means, I don’t have to explain because I have watched your lecture.

https://www.youtube.com/watch?v=47-deZ8uT-w

Your video is on my You-Tube channel and has had 2.2k views so far. I put it up on my You-Tube channel a couple of years ago. When I posted your video it was from my interest in collapse, environmentalism and such. I forgot how I found it and I did not Google who you were until today. My interest in building the Dynamo simulated renewed my interest, and I re-watched the video.

I have Forester’s Book and The Dynamo Users Manual. An original copy of Limits To Growth is supposed to be in the mail on the way to me. The second edition is of little use to the project.

I would greatly appreciate getting my paws on a copy of the original LTG card deck if you have a copy. My goal is to have a page on my own website where the original card deck can be run, and where a user can load their own files. I want users to be able to write their own models.

Thank You,
Keith Hayes

]]> Comment on AI and the end of programming by johngag http://bit-player.org/2023/ai-and-the-end-of-programming/comment-page-1#comment-470790 johngag Thu, 21 Mar 2024 17:06:46 +0000 http://bit-player.org/?p=2496#comment-470790 I really do not think the AI models will fizzle. They are here to stay and in my opinion it is just a matter of time before Matt Welsh is correct. Excellent article. I really do not think the AI models will fizzle. They are here to stay and in my opinion it is just a matter of time before Matt Welsh is correct. Excellent article.

]]> Comment on AI and the end of programming by Dale Vivian Ross http://bit-player.org/2023/ai-and-the-end-of-programming/comment-page-1#comment-463552 Dale Vivian Ross Tue, 19 Dec 2023 00:11:28 +0000 http://bit-player.org/?p=2496#comment-463552 I asked ChatGPT to produce acrostic poems and it was a task. It kept correcting itself missing letters, to updated versions still missing letters. Drove me nuts-er. I asked ChatGPT to produce acrostic poems and it was a task. It kept correcting itself missing letters, to updated versions still missing letters. Drove me nuts-er.

]]> Comment on AI and the end of programming by David Speyer http://bit-player.org/2023/ai-and-the-end-of-programming/comment-page-1#comment-454160 David Speyer Thu, 05 Oct 2023 15:30:47 +0000 http://bit-player.org/?p=2496#comment-454160 I really liked this piece for its balance and clarity. But, of course, what drives me to comment is a nitpick. "Shakespeare wrote 10 plays about kings of England, even though he had never ... met one." This seems unlikely to me. James I named Shakespeare and his company "The King's Men", patronized them, named them Grooms of the Chamber and invited them to perform in his court on dozens of occasions. I can't find a particular occasion on which I am sure that Shakespeare met James I, but it seems likely to have happened. I really liked this piece for its balance and clarity. But, of course, what drives me to comment is a nitpick.

“Shakespeare wrote 10 plays about kings of England, even though he had never … met one.”

This seems unlikely to me. James I named Shakespeare and his company “The King’s Men”, patronized them, named them Grooms of the Chamber and invited them to perform in his court on dozens of occasions. I can’t find a particular occasion on which I am sure that Shakespeare met James I, but it seems likely to have happened.

]]> Comment on Riding the Covid coaster by Alexey Nedosekin http://bit-player.org/2021/riding-the-covid-coaster/comment-page-1#comment-453384 Alexey Nedosekin Wed, 27 Sep 2023 10:58:04 +0000 http://bit-player.org/?p=2359#comment-453384 Brian, hi. You have got really wonderfool result. Could You please provide me with coefficients of Fourier approximation for Figure 7? Thank You in advance. Alexey Brian, hi. You have got really wonderfool result. Could You please provide me with coefficients of Fourier approximation for Figure 7? Thank You in advance. Alexey

]]> Comment on AI and the end of programming by not feeling very nymous http://bit-player.org/2023/ai-and-the-end-of-programming/comment-page-1#comment-452958 not feeling very nymous Mon, 25 Sep 2023 14:01:49 +0000 http://bit-player.org/?p=2496#comment-452958 A small correction to this really nice piece: the "attention" mechanism actually only has a fixed and finite range it looks back. (It replaces the vector representations of a token with a weighted average of all the preceding vectors in that context length, with more weight given to more similar vectors [https://arxiv.org/abs/1908.11775].) Typically this is a few thousand tokens, but it's still just a finite-order Markov model. Of course it doesn't keep track of a separate next-token distribution for each umpteen-thousand-token context, the way a simple n-gram model would. The magic of "attention" lies in cleverly sharing information across similar long-but-finite contexts, not in allowing unlimited-length dependence. None of this is especially easy to decrypt from the original papers, but [https://arxiv.org/abs/2207.09238] helpfully provides (correct!) pseudocode. A small correction to this really nice piece: the “attention” mechanism actually only has a fixed and finite range it looks back. (It replaces the vector representations of a token with a weighted average of all the preceding vectors in that context length, with more weight given to more similar vectors [https://arxiv.org/abs/1908.11775].) Typically this is a few thousand tokens, but it’s still just a finite-order Markov model. Of course it doesn’t keep track of a separate next-token distribution for each umpteen-thousand-token context, the way a simple n-gram model would. The magic of “attention” lies in cleverly sharing information across similar long-but-finite contexts, not in allowing unlimited-length dependence.

None of this is especially easy to decrypt from the original papers, but [https://arxiv.org/abs/2207.09238] helpfully provides (correct!) pseudocode.

]]> Comment on AI and the end of programming by Simon Willison http://bit-player.org/2023/ai-and-the-end-of-programming/comment-page-1#comment-451716 Simon Willison Thu, 14 Sep 2023 11:16:22 +0000 http://bit-player.org/?p=2496#comment-451716 I tried Code Interpreter (now renamed to "Advanced Data Analysis" which I think is an even worse name) with the following: <blockquote>Write a Python program to solve five letter word ladders like Donald Knuth would. Use /etc/words or similar if you have access to it. Show it working on PEACH to GRASP</blockquote> It appeared to know the Knuth solution and spat out a working version of it. I had to upload a word list to it (you can upload files to Code Interpreter), after which I got some convincing results. It also used a neat optimization for building the graph, after it timed out the first time. Details in this transcript: https://chat.openai.com/share/c2b2538e-4d8b-40e2-a603-b9808b932000 Sharing this here mainly to illustrate my own perspective on LLMs in general and Code Interpreter specifically: they're deceptively difficult to use effectively. If you have deep knowledge of both the subject area and what the tools themselves are capable of you can get remarkable results out of them, but it's much easier and more common to get disappointing results from them. I tried Code Interpreter (now renamed to “Advanced Data Analysis” which I think is an even worse name) with the following:

Write a Python program to solve five letter word ladders like Donald Knuth would. Use /etc/words or similar if you have access to it. Show it working on PEACH to GRASP

It appeared to know the Knuth solution and spat out a working version of it. I had to upload a word list to it (you can upload files to Code Interpreter), after which I got some convincing results.

It also used a neat optimization for building the graph, after it timed out the first time. Details in this transcript: https://chat.openai.com/share/c2b2538e-4d8b-40e2-a603-b9808b932000

Sharing this here mainly to illustrate my own perspective on LLMs in general and Code Interpreter specifically: they’re deceptively difficult to use effectively. If you have deep knowledge of both the subject area and what the tools themselves are capable of you can get remarkable results out of them, but it’s much easier and more common to get disappointing results from them.

]]> Comment on AI and the end of programming by Doug http://bit-player.org/2023/ai-and-the-end-of-programming/comment-page-1#comment-451696 Doug Thu, 14 Sep 2023 08:23:06 +0000 http://bit-player.org/?p=2496#comment-451696 I feel like no matter whether a particular LLM can solve a particular problem, we finally have all the pieces that we will need to build something to solve any problem a human can. Before LLMs, any problem that required understanding English was simply out of the question-- only simpler invented languages could be used. Now, that's no longer a limitation. Current LLMs are not good enough at reasoning and abstract thinking, but we already knew how to do that kind of thing with computers. It was the soft stuff we couldn't figure out. Analogies? Applying ideas to new domains? LLMs can do that, it's just that graph structures and letter puzzles were still too hard for the current system. So I think it's just a matter of time before the problems you bring up here are solved, and my guess is that time will be measured in years, not decades. I feel like no matter whether a particular LLM can solve a particular problem, we finally have all the pieces that we will need to build something to solve any problem a human can. Before LLMs, any problem that required understanding English was simply out of the question– only simpler invented languages could be used. Now, that’s no longer a limitation. Current LLMs are not good enough at reasoning and abstract thinking, but we already knew how to do that kind of thing with computers. It was the soft stuff we couldn’t figure out. Analogies? Applying ideas to new domains? LLMs can do that, it’s just that graph structures and letter puzzles were still too hard for the current system. So I think it’s just a matter of time before the problems you bring up here are solved, and my guess is that time will be measured in years, not decades.

]]> Comment on AI and the end of programming by felipe http://bit-player.org/2023/ai-and-the-end-of-programming/comment-page-1#comment-451685 felipe Thu, 14 Sep 2023 07:35:25 +0000 http://bit-player.org/?p=2496#comment-451685 maybe reading on BPE Encoding will provide explanation as to why the ladder game is hard for the AI, perhaps a game made out of BPE Encodings (like, latin prefixes and suffixes) would be more appropriate as a test for the machine maybe reading on BPE Encoding will provide explanation as to why the ladder game is hard for the AI, perhaps a game made out of BPE Encodings (like, latin prefixes and suffixes) would be more appropriate as a test for the machine

]]> Comment on AI and the end of programming by Brian http://bit-player.org/2023/ai-and-the-end-of-programming/comment-page-1#comment-451670 Brian Thu, 14 Sep 2023 05:50:54 +0000 http://bit-player.org/?p=2496#comment-451670 I love the use of word ladders as both a puzzle to solve and a programming exercise. It's instructive how the program's output moves from sensible to off-the-rails so fluidly: indeed, both are drawn from the same place. I can't help comparing this to the output a friend of mine got from ChatGPT when he challenged it to write a program in INTERCAL (link in my name). Though the results were far more frivolous than yours, I can see similar themes in where and how its output goes wrong. Unfortunately, the one thing that these programs seem to be striving to improve at is <em>sounding</em> convincing. To me the worry is not that the programs will get good enough to replace human programmers, but rather that they will anyway. I love the use of word ladders as both a puzzle to solve and a programming exercise. It’s instructive how the program’s output moves from sensible to off-the-rails so fluidly: indeed, both are drawn from the same place.

I can’t help comparing this to the output a friend of mine got from ChatGPT when he challenged it to write a program in INTERCAL (link in my name). Though the results were far more frivolous than yours, I can see similar themes in where and how its output goes wrong.

Unfortunately, the one thing that these programs seem to be striving to improve at is sounding convincing. To me the worry is not that the programs will get good enough to replace human programmers, but rather that they will anyway.

]]> Comment on Revised and Updated by Nicholas Whitmarsh http://bit-player.org/2014/revised-and-updated/comment-page-1#comment-418048 Nicholas Whitmarsh Sat, 21 Jan 2023 06:58:31 +0000 http://bit-player.org/?p=1667#comment-418048 Infra- means "below;" so the infrastructure is the "underlying structure" of a country and its economy, the fixed installations that it needs in order to function. These include roads, bridges, dams, the water and sewer systems, railways and subways, airports, and harbors Infra- means “below;” so the infrastructure is the “underlying structure” of a country and its economy, the fixed installations that it needs in order to function. These include roads, bridges, dams, the water and sewer systems, railways and subways, airports, and harbors

]]> Comment on Glauber’s dynamics by idioticbaka1824 http://bit-player.org/2019/glaubers-dynamics/comment-page-1#comment-414218 idioticbaka1824 Sat, 24 Dec 2022 17:42:48 +0000 http://bit-player.org/?p=2108#comment-414218 This article was a great read! Plenty of simple explanations, and discussion of details I don't usually see elsewhere. Thanks! Also, I'm not sure if it was like this in the original source, but there's a typo in the quote at the end: "time-dependent <strong>h</strong>ehavior of statistical systems" This article was a great read! Plenty of simple explanations, and discussion of details I don’t usually see elsewhere. Thanks!
Also, I’m not sure if it was like this in the original source, but there’s a typo in the quote at the end:
“time-dependent hehavior of statistical systems”

]]> Comment on The Middle of the Square by Bill Satzer http://bit-player.org/2022/the-middle-of-the-square/comment-page-1#comment-403304 Bill Satzer Sat, 01 Oct 2022 19:07:14 +0000 http://bit-player.org/?p=2480#comment-403304 I have to wonder how much damage poor random number generators have caused. Years ago, when I worked for a supercomputer manufacturer, the standard speed test for supercomputing used a linear congruential generator with too short a period to generate 100 x 100 matrices for a test of the speed of solving linear systems. The speed test would generate many linear systems like this and collect the overall solution time. Of course, the systems that were generated were generally near singular, and should have triggered exception handling. But that wasn't part of the test. So the identification of the fastest supercomputer for some years involved finding many meaningless solutions to ill-conditioned systems. I have to wonder how much damage poor random number generators have caused. Years ago, when I worked for a supercomputer manufacturer, the standard speed test for supercomputing used a linear congruential generator with too short a period to generate 100 x 100 matrices for a test of the speed of solving linear systems. The speed test would generate many linear systems like this and collect the overall solution time.

Of course, the systems that were generated were generally near singular, and should have triggered exception handling. But that wasn’t part of the test. So the identification of the fastest supercomputer for some years involved finding many meaningless solutions to ill-conditioned systems.

]]> Comment on The Middle of the Square by Gerry Myerson http://bit-player.org/2022/the-middle-of-the-square/comment-page-1#comment-401134 Gerry Myerson Sun, 18 Sep 2022 09:28:25 +0000 http://bit-player.org/?p=2480#comment-401134 I'm not convinced of the statements about the logistic map, f(x) = 4 x (1 - x). One, I believe this map is known to have periodic points with period n for all n, and for all n > 1 those periodic points are irrational. Two, "chaotic" is not the same as "uniformly distributed". The part of the unit interval that gets mapped to, say, [.9, 1], has greater measure than the part mapped to, say, [0, .1], so I would expect that the iteration spends more time in [.9, 1] than in [0, .1]. I’m not convinced of the statements about the logistic map, f(x) = 4 x (1 - x). One, I believe this map is known to have periodic points with period n for all n, and for all n > 1 those periodic points are irrational. Two, “chaotic” is not the same as “uniformly distributed”. The part of the unit interval that gets mapped to, say, [.9, 1], has greater measure than the part mapped to, say, [0, .1], so I would expect that the iteration spends more time in [.9, 1] than in [0, .1].

]]> Comment on World3, the public beta by Tom Rust http://bit-player.org/2012/world3-the-public-beta/comment-page-1#comment-400933 Tom Rust Fri, 16 Sep 2022 19:15:49 +0000 http://bit-player.org/?p=1172#comment-400933 What if you tested the model with more historical data that could run out a longer time to test the predictions. If you use, say, ancient Rome. Choose a date of some importance, say 100 CE, well into the peace and prosperity of the Pax Romana but before the distress of the third century. Much like Ian Morris' works, the <em>Measure of Civilization</em> and <em>Why the West Rules for Now</em>, some quantifiable data can be gleaned from the historical and archaeological record that could be used as input for the model. Then the simulation could be run out in different scenarios for a 100, 200, 300, 400, and 500 year predictive models. Modeling the different scenarios could then be cross referenced with known historical outcomes. It might be just as useful for historians/archaeologists as it would a test for the model as it could provide new questions not yet conceived which might beg answers. What if you tested the model with more historical data that could run out a longer time to test the predictions. If you use, say, ancient Rome. Choose a date of some importance, say 100 CE, well into the peace and prosperity of the Pax Romana but before the distress of the third century. Much like Ian Morris’ works, the Measure of Civilization and Why the West Rules for Now, some quantifiable data can be gleaned from the historical and archaeological record that could be used as input for the model. Then the simulation could be run out in different scenarios for a 100, 200, 300, 400, and 500 year predictive models. Modeling the different scenarios could then be cross referenced with known historical outcomes. It might be just as useful for historians/archaeologists as it would a test for the model as it could provide new questions not yet conceived which might beg answers.

]]>