AI - The artificial intelligence thread

[GORDO]

The parallel computations used by GPUs in ML are the same kinds of parallel computations used by GPUs in graphic processing: Linear Algebra. They're only doing really big matrix multiplications.

I don't know much about quantum computing but I don't think it would achieve gains in the speed or volume of these type of computations which are straight forward, the edge quantum computing lies in other problem spaces, I think.

But I could be wrong, maybe quantum computing would make training a deep learning model orders of magnitude faster or maybe open the way for other types of parameter fitting or maybe (I'm just wildly guessing rn) make the parameters diffuse and be a probability distribution rather than fixed real numbers, models would be then stochastic rather than deterministic.

[iococoi]

what could possibly go wrong?

https://www.vice.com/en/article/qjvb4x/palantir-demos-ai-to-fight-wars-but-says-it-will-be-totally-ethical-dont-worry-about-it

[iococoi]

Quote

"The world of the future will be an ever more demanding struggle against the limitations of our intelligence, not a comfortable hammock in which we can lie down to be waited upon by our robot slaves."

— Norbert Wiener, God & Golem, Inc. (Cambridge: MIT Press, 1964) p. 69.

 

[cichlisuite]

1 hour ago, iococoi said:

what could possibly go wrong?

https://www.vice.com/en/article/qjvb4x/palantir-demos-ai-to-fight-wars-but-says-it-will-be-totally-ethical-dont-worry-about-it

the future wars will be narrated by a vocal frying valley girl

"the allied forces LiTeRallY outflanked enemy positions, and the enemy was like, "oMG, what's even happening?!", but it was too late." #maneuverwarfare #aiassistedbattlefields

Edited May 6, 2023 by cichlisuite

[cichlisuite]

On 5/5/2023 at 12:55 AM, GORDO said:

I see this as a common misunderstanding. These models are not databases, they don't 'look up' stuff. They're enormously complex formulas whose parameters are 'fitted' with data towards a target objective. It's no different conceptually than finding the best intercept and slope to fit a line through a bunch of points in a plane. The difference is that instead of two parameters you're finding the fit for billions of parameters in an humanly intractable formula. It's statistics on steroids, not databases.

This is why we can't have 'pseudo code' for how they produce their outputs and why we say we don't 'understand' how they work. While we do know *exactly* what the 'formula' is and we know the parameters, it's so immensely big and complicated that there's no point in trying to extract insight from it. There are tho techniques to gain partial understanding on how variation of their inputs produce variations on the outputs and so on.

Whenever I see the argument that they don't posses knowledge I'd argue back that whatever test you can come up for a human to prove knowledge, this thing can do it too.

i didn't say they were databases, i asked if there are components to the whole system, which must(?) include databases... i mean there must be something structured that feeds into this neural network (for learning purposes), it's not gibberish (or is it?) so my question was (is), does it need a structured and organized data set to be able to learn to think of  a relevant output, or you can feed it random strings and integers which are only chunks of real data, and from it, it is able to make sense of it by completing the logical gaps by itself? again, i'm asking about the learning process here.

and another thing: once a version of chatgpt is 'taught' something, is this knowledge referenced at some point when the user writes a question in the prompt, or is it performing the entire computation from scratch when the user writes a question in the prompt? i'm sorry if i'm prying, or sounding thick for not getting it...you can ignore me if i annoy you.

Quote

This is why we can't have 'pseudo code' for how they produce their outputs and why we say we don't 'understand' how they work. While we do know *exactly* what the 'formula' is and we know the parameters, it's so immensely big and complicated that there's no point in trying to extract insight from it. There are tho techniques to gain partial understanding on how variation of their inputs produce variations on the outputs and so on.

so the neural network by ways of structure and recursion is making 'the magic'? do you know what kind of operations (functions) are assigned to an individual node in that structure?

 

 

[Freak of the week]

On 5/1/2023 at 1:41 PM, auxien said:

“When you see something that is technically sweet, you go ahead and do it.”

full quote: "When you see something that is technically sweet, you go ahead and do it and you argue about what to do about it only after you have had your technical success."

well put

[vkxwz]

6 hours ago, cichlisuite said:

So the neural network by ways of structure and recursion is making 'the magic'? do you know what kind of operations (functions) are assigned to an individual node in that structure?

 

The neural networks are just chains of matrix multiplication, fairly straightforward math so we understand what operations are being done. But the amount of computation happening is so massive and the training process creates emergent structures in the operations(by modifying them automatically in the training process, gradually over time). The issue is that it's quite difficult to identify and understand this emergent structure even though we can see that it works and know every step of the math done to compute the output. it's like trying trying to understand how a human brain generates thoughts just based on data that is the firing patterns of all neurons in the brain at all times. There is a whole science around how to set up the neural networks like an environment that the emergent functions can "evolve" in, but much less understanding about the nature of those functions.

[GORDO]

6 hours ago, cichlisuite said:

i didn't say they were databases, i asked if there are components to the whole system, which must(?) include databases... i mean there must be something structured that feeds into this neural network (for learning purposes), it's not gibberish (or is it?) so my question was (is), does it need a structured and organized data set to be able to learn to think of  a relevant output, or you can feed it random strings and integers which are only chunks of real data, and from it, it is able to make sense of it by completing the logical gaps by itself? again, i'm asking about the learning process here.

and another thing: once a version of chatgpt is 'taught' something, is this knowledge referenced at some point when the user writes a question in the prompt, or is it performing the entire computation from scratch when the user writes a question in the prompt? i'm sorry if i'm prying, or sounding thick for not getting it...you can ignore me if i annoy you.

so the neural network by ways of structure and recursion is making 'the magic'? do you know what kind of operations (functions) are assigned to an individual node in that structure?

 

 

It needs structured data yes, you have to provide it with the "right" answers and feed it a ton of them. Afaik chatgpt is a 'next best word' model, so the right answer for that is just the word that succeeded some other chain of words.

More generally as I was saying before what machine learning models do is fit a function: f(X) = y. and the process of learning is finding what exactly this 'f' is. For it to 'learn' you feed it examples of X and y, and the process consists of evaluating and adjusting iteratively a bunch of different functions until the 'error' is minimized in some way. Again,  think of finding the best intercept and slope, it is truly no more different than this, only exploded in complexity.

Afaik or unless there's some other magic in chatgpt, it's doing the computation by scratch in each session, but everything that's been said in the session is fed as input and that's why you can 'correct' or ask for changes. 

Units in NN can have pretty much any operation in them as long as it is differentiable (talking calculus here) but there are some common ones, they're refered to as activation functions, you're better off looking them up in wiki than me trying to list them.

 

[GORDO]

21 hours ago, Summon Dot E X E said:

Structured and unstructured learning both exist. Structured means the data is labeled. Unstructured means it isn't.

The models themselves are "black boxes". Researchers have gained some visibility into it, but the high level of dimensionality limits human understanding. Emergent abilities in LLM continue to surprise people as well, as has been discussed here... abilities the model wasn't specifically trained to have but they nevertheless possess.

No, you're talking about supervised and unsupervised learning. Both require data that's structured in some way to fit the problem at hand.

In this field we say unstructured data we are often talking about data that isn't a table such as images, sounds or words.

Edited May 7, 2023 by GORDO

[NewSchoolScience]

 

[iococoi]

guud rheed ...

https://www.theguardian.com/commentisfree/2023/may/08/ai-machines-hallucinating-naomi-klein

[cichlisuite]

3 hours ago, iococoi said:

guud rheed ...

https://www.theguardian.com/commentisfree/2023/may/08/ai-machines-hallucinating-naomi-klein

This is very well written, and I thoroughly agree. But will we be able to stop the giant grind stone?

[zero]

6 hours ago, cichlisuite said:

This is very well written, and I thoroughly agree. But will we be able to stop the giant grind stone?

no, we won't. the humans at the top of the chain running the planet don't like to hear the truth, because it doesn't fit in with their delusional view of reality. I agree the author of that article really nailed it on just about all points. this one especially so - 

Quote

There is a world in which generative AI, as a powerful predictive research tool and a performer of tedious tasks, could indeed be marshalled to benefit humanity, other species and our shared home. But for that to happen, these technologies would need to be deployed inside a vastly different economic and social order than our own, one that had as its purpose the meeting of human needs and the protection of the planetary systems that support all life.

imo you always have to look at the psychology of any situation involving the human decision making process, take into account individual's personal motivations when they are faced with important choices to make. this slow AI roll out is no different. it is being pushed out in an entirely capitalistically way, to a global society already totally addicted to the latest and greatest technology, by profit motivated mega rich humans. and there is no end game planning here. just keep building it, keep competing with other companies, and see what eventually happens. 

and kinda ironic that on an IDM forum, there are a lot of folks that seem to recognize this barrel of a loaded gun the collective us are starring down. 20 or so years ago when AI was still very far fetched, a lifetimes away sorta thing, I was all about the idea that AI should replace humans. now I dream of a society where computers were never invented. where there are no screens, no phones to stare at, telling us what to do. 

[cichlisuite]

On 5/6/2023 at 4:27 PM, GORDO said:

It needs structured data yes, you have to provide it with the "right" answers and feed it a ton of them. Afaik chatgpt is a 'next best word' model, so the right answer for that is just the word that succeeded some other chain of words.

More generally as I was saying before what machine learning models do is fit a function: f(X) = y. and the process of learning is finding what exactly this 'f' is. For it to 'learn' you feed it examples of X and y, and the process consists of evaluating and adjusting iteratively a bunch of different functions until the 'error' is minimized in some way. Again,  think of finding the best intercept and slope, it is truly no more different than this, only exploded in complexity.

Afaik or unless there's some other magic in chatgpt, it's doing the computation by scratch in each session, but everything that's been said in the session is fed as input and that's why you can 'correct' or ask for changes. 

Units in NN can have pretty much any operation in them as long as it is differentiable (talking calculus here) but there are some common ones, they're refered to as activation functions, you're better off looking them up in wiki than me trying to list them.

 

Thanks that was informative. Could you say it's a game of elimination of a sort, in a way that it first 'explodes' in contexts and possibilities, and then slowly starts picking up the strongest threads, and the array of contexts and possibilities shrinks, until only one possible solution remains...?

[cichlisuite]

33 minutes ago, zero said:

no, we won't. the humans at the top of the chain running the planet don't like to hear the truth, because it doesn't fit in with their delusional view of reality. I agree the author of that article really nailed it on just about all points. this one especially so - 

imo you always have to look at the psychology of any situation involving the human decision making process, take into account individual's personal motivations when they are faced with important choices to make. this slow AI roll out is no different. it is being pushed out in an entirely capitalistically way, to a global society already totally addicted to the latest and greatest technology, by profit motivated mega rich humans. and there is no end game planning here. just keep building it, keep competing with other companies, and see what eventually happens. 

and kinda ironic that on an IDM forum, there are a lot of folks that seem to recognize this barrel of a loaded gun the collective us are starring down. 20 or so years ago when AI was still very far fetched, a lifetimes away sorta thing, I was all about the idea that AI should replace humans. now I dream of a society where computers were never invented. where there are no screens, no phones to stare at, telling us what to do. 

agreed. one tries to be positive, and hope for the best, but the reality is that we are helpless until the concentrated power of super rich entities is equaled by a power of coherent mass initiative that will have to act as a counterweight, continually sustained in order to maintain the balance of powers (not just campaign-like, single purpose mass initiative, but a continuous one). not just for ai, but for many other pressing things.

the next best option (and that's saying something), and far more probable (unfortunately), is to continue to be a reactionary force, always lagging behind with legislature initiatives, far later when the damage is long done, and our ways of life accepted it and adopted it with another source of anxiety, while the legislature only barely contains the fringes of capital rampage.

[GORDO]

4 hours ago, cichlisuite said:

Thanks that was informative. Could you say it's a game of elimination of a sort, in a way that it first 'explodes' in contexts and possibilities, and then slowly starts picking up the strongest threads, and the array of contexts and possibilities shrinks, until only one possible solution remains...?

It's an iterative process where the error is minimized in each step slightly by adjusting the parameters of the function.

Again think of slope and intercept. I start with random pair (s0, i0) so f(x)=i0+s0x I check how this function fits an observation from the problem I'm trying to predict and adjust the parameters based on the error, so now I got (s1, i1) and so f(x) = i1 + s1x and I keep repeating this until I run out of data or no longer gain any significant reduction in the error. Now imagine instead of a pair of parameters I have billions and f is a stupidly comprehensive formula that allows for any kind of interaction between its variables, and that the problem is something more interesting than fitting a line through points.

Interestingly there does not need to be an 'unique solution' since both the initialization and method for adjusting parameters could yield different overall results, the result being the specific parameters that yielded the least error.

[zero]

good recap of where we're at in the AI philosophical dilemma - 

Threats by artificial intelligence to human health and human existence

https://gh.bmj.com/content/8/5/e010435

 

lol here's a good idea for Skynet! it'll be able to wipe out large swaths of the population with ease:

Quote

For example, it is possible for a million tiny drones equipped with explosives, visual recognition capacity and autonomous navigational ability to be contained within a regular shipping container and programmed to kill en masse without human supervision.

 

[prdctvsm]

Skyborg 

 

[ignatius]

4 hours ago, prdctvsm said:

Skyborg 

 

can we just get healthcare and rent control.. maybe daycare for kids and fix 3 million potholes... school lunch for kids... pay raise for teachers.. or any of the thousands of things that would make life better for people here in this fucking upside down tardistan of a country. 

[prdctvsm]

9 minutes ago, ignatius said:

can we just get healthcare and rent control.. maybe daycare for kids and fix 3 million potholes... school lunch for kids... pay raise for teachers.. or any of the thousands of things that would make life better for people here in this fucking upside down tardistan of a country. 

 computah sez "skyborg" 

[GORDO]

posting this here as a bookmark

https://www.lepoint.fr/sciences-nature/yuval-harari-sapiens-versus-yann-le-cun-meta-on-artificial-intelligence-11-05-2023-2519782_1924.php

[Alcofribas]

19 hours ago, GORDO said:

posting this here as a bookmark

https://www.lepoint.fr/sciences-nature/yuval-harari-sapiens-versus-yann-le-cun-meta-on-artificial-intelligence-11-05-2023-2519782_1924.php

this is more Alco Trigger Point stuff. YLC making poor definitions, constantly asserting "in the future it will be like this," side-stepping all criticism with "we already have problems." weak shit.

[Real Human Bean]

On 5/11/2023 at 9:15 PM, Summon Dot E X E said:

I'm hoping to get a chance to use this soon: https://www.linum.ai/

This just looks like a front end for Deforum. Why not use that instead?

[mcbpete]

7 minutes ago, Real Human Bean said:

This just looks like a front end for Deforum. Why not use that instead?

Every AI video looks like every AI video. It's the bullet-time of this decade to immediately date videos

[Real Human Bean]

1 minute ago, mcbpete said:

Every AI video looks like every AI video. It's the bullet-time of this decade to immediately date videos

That's not true at all though...Deforum, Warp Fusion, Kaliber, Maua all have distinct looks and that's just Stable Diffusion based ones. Then there's Disco Diffusion, VQGAN, Pytti...then the text to video stuff like ModelScope and Gen-2.