Scientists invented a fake disease. AI told people it was real

4,324 Views | 50 Replies | Last: 6 days ago by agracer
one safe place
How long do you want to ignore this user?
DannyDuberstein said:

In my head, I always read infinity ag's post in the voice of an over-caffeinated Indian dude, and it's fairly entertaining

Yeah, and I read it with the Indian accent. That might make me a berry, berry bad man.
infinity ag
How long do you want to ignore this user?
Logos Stick said:

In your example though, if AI did not exist, you still have vets who are now refusing to prescribe those two meds together, which you've said is completely wrong.

You can always ground AI to prevent answers based on garbage, i.e. completely ignore this source when generating an answer, or use these sources and only those sources, nothing else.

Also, there are models trained specifically on valid medical info - PubMed literature, clinical notes, medical textbooks, and knowledge bases like UMLS, rather than general internet data .

That would not help in your example potentially, but would certainly negate the issue brought up by the OP.


Yes, you can ground the AI system to focus only on acceptable literature and not on what the model is trained on. However, I don't think most people would have access to these systems which need some work to build. (called Retrieval Augmented Generation Architecture) Most would just use ChatGPT or Copilot or something to get quick answers for free/cheap.
I am not aware if there are models that are just trained on Medical literature, that may be what some company needs to look into.

My point is that China and some other countries good at information warfare will use this to taint AI outputs with outcomes that they want. That will be the next frontier.
infinity ag
How long do you want to ignore this user?
MouthBQ98 said:

I agree with you in general but I do think in the short term, upper management are gambling they can cut work force competitively with other businesses to keep pace with improved margins on financial reports and that the pace of AI productivity improvements will close the gap before things fall apart internally, or be able to sweep in and clean up the mess. Short term decision making is still dominating board rooms and C suites and they're going to want to show competitive employment expense savings.

The irony is that the upper management levels will be the most easily replaced element in a business by AI but there's not a chance in hell that gets implemented.


You are right, I agree with you here. The current volatility we are seeing is from this. I feel it will subside in some time and get to equilibrium at some point, no one knows when.
infinity ag
How long do you want to ignore this user?
Logos Stick said:

OP, one of these days you will learn how AI works. I'm pulling for you.


Let's take those courses together!
UntoldSpirit
How long do you want to ignore this user?
AG
infinity ag said:

Deputy Travis Junior said:

It's supposed to be surprising that when an AI system is trained on tainted data, it spits out bogus answers?

Wow I guess.


Most people are not as knowledgeable as you are about AI. Many, even on this board, think that AI is some kind of magical thing. They don't know about garbage in - garbage out.

I asked AI a question about my taxes, and it made an obvious arithmetic error. It went on this thorough explanation of everything and made perfect sense except the numbers were obviously wrong. Like 2+2 = 5 stuff. I almost didn't catch it at first, since it seemed to understand the situation so well.

So i confronted it and said hey you made an obvious math error, and it said oh yeah, I guess you're right. I got confused - here are the real numbers.

It's obvious to most people at this point, IMO, that you can't just rely on AI without some critical analysis and consideration of its answer. It is quite human.
Logos Stick
How long do you want to ignore this user?
infinity ag said:

Logos Stick said:

In your example though, if AI did not exist, you still have vets who are now refusing to prescribe those two meds together, which you've said is completely wrong.

You can always ground AI to prevent answers based on garbage, i.e. completely ignore this source when generating an answer, or use these sources and only those sources, nothing else.

Also, there are models trained specifically on valid medical info - PubMed literature, clinical notes, medical textbooks, and knowledge bases like UMLS, rather than general internet data .

That would not help in your example potentially, but would certainly negate the issue brought up by the OP.


Yes, you can ground the AI system to focus only on acceptable literature and not on what the model is trained on. However, I don't think most people would have access to these systems which need some work to build. (called Retrieval Augmented Generation Architecture) Most would just use ChatGPT or Copilot or something to get quick answers for free/cheap.
I am not aware if there are models that are just trained on Medical literature, that may be what some company needs to look into.

My point is that China and some other countries good at information warfare will use this to taint AI outputs with outcomes that they want. That will be the next frontier.


No medical professional should be using public LLMs trained on anything and everything to do medical diagnosis. Anyone who is doing that doesn't understand AI. Take some courses or watch some videos if you truly want to understand it.
infinity ag
How long do you want to ignore this user?
Logos Stick said:

infinity ag said:

Logos Stick said:

In your example though, if AI did not exist, you still have vets who are now refusing to prescribe those two meds together, which you've said is completely wrong.

You can always ground AI to prevent answers based on garbage, i.e. completely ignore this source when generating an answer, or use these sources and only those sources, nothing else.

Also, there are models trained specifically on valid medical info - PubMed literature, clinical notes, medical textbooks, and knowledge bases like UMLS, rather than general internet data .

That would not help in your example potentially, but would certainly negate the issue brought up by the OP.


Yes, you can ground the AI system to focus only on acceptable literature and not on what the model is trained on. However, I don't think most people would have access to these systems which need some work to build. (called Retrieval Augmented Generation Architecture) Most would just use ChatGPT or Copilot or something to get quick answers for free/cheap.
I am not aware if there are models that are just trained on Medical literature, that may be what some company needs to look into.

My point is that China and some other countries good at information warfare will use this to taint AI outputs with outcomes that they want. That will be the next frontier.


No medical professional should be using public LLMs trained on anything and everything to do medical diagnosis. Anyone who is doing that doesn't understand AI. Take some courses or watch some videos if you truly want to understand it.


I 100% agree with you, but "should not be" is different from "I just did". Many doctors won't spend the time to research through literature which takes time and effort and reduces their billing.

Everyone wants an easy way to do things and the hype will guide them towards using it.
japantiger
How long do you want to ignore this user?
S
infinity ag said:

Logos Stick said:

In your example though, if AI did not exist, you still have vets who are now refusing to prescribe those two meds together, which you've said is completely wrong.

You can always ground AI to prevent answers based on garbage, i.e. completely ignore this source when generating an answer, or use these sources and only those sources, nothing else.

Also, there are models trained specifically on valid medical info - PubMed literature, clinical notes, medical textbooks, and knowledge bases like UMLS, rather than general internet data .

That would not help in your example potentially, but would certainly negate the issue brought up by the OP.


Yes, you can ground the AI system to focus only on acceptable literature and not on what the model is trained on. However, I don't think most people would have access to these systems which need some work to build. (called Retrieval Augmented Generation Architecture) Most would just use ChatGPT or Copilot or something to get quick answers for free/cheap.
I am not aware if there are models that are just trained on Medical literature, that may be what some company needs to look into.

My point is that China and some other countries good at information warfare will use this to taint AI outputs with outcomes that they want. That will be the next frontier.

I thought this thread was all about the Wuhan Flu...then I noticed it was the usual InfinityAg Luddite post.

No medical practitioner is going to us a generic LLM for medical/clinical advice/treatments.

Like with anything AI related, domain specific AI's are what is driving ROI for AI and has been for 3 decades (yes, practical AI has been around a long time).

There are at least 5 domain specific medical AI's in use currently that I am aware of that have been built for specific purposes...for example, PathAI for pathology, AIDoc for radiology, AMBOSS for clinical use, MEDGemma for imaging, etc.

The issue with your use of only "acceptable literature" in LLMs, is who decides what is acceptable. If the answer is the Leftist Intelligentsia, then no thanks. Like with all automation, domain specific uses will provide a greater degree of accuracy and will find value in the marketplace.

infinity ag
How long do you want to ignore this user?
UntoldSpirit said:

infinity ag said:

Deputy Travis Junior said:

It's supposed to be surprising that when an AI system is trained on tainted data, it spits out bogus answers?

Wow I guess.


Most people are not as knowledgeable as you are about AI. Many, even on this board, think that AI is some kind of magical thing. They don't know about garbage in - garbage out.

I asked AI a question about my taxes, and it made an obvious arithmetic error. It went on this thorough explanation of everything and made perfect sense except the numbers were obviously wrong. Like 2+2 = 5 stuff. I almost didn't catch it at first, since it seemed to understand the situation so well.

So i confronted it and said hey you made an obvious math error, and it said oh yeah, I guess you're right. I got confused - here are the real numbers.

It's obvious to most people at this point, IMO, that you can't just rely on AI without some critical analysis and consideration of its answer. It is quite human.


I have experienced the same types of issues, but I disagree that it is obvious to many people. Which is where the risk is. Most don't have the time or inclination to check or verify. I see that all the time even at work, people present AI generated slop without review and the errors are visible.
infinity ag
How long do you want to ignore this user?
japantiger said:

infinity ag said:

Logos Stick said:

In your example though, if AI did not exist, you still have vets who are now refusing to prescribe those two meds together, which you've said is completely wrong.

You can always ground AI to prevent answers based on garbage, i.e. completely ignore this source when generating an answer, or use these sources and only those sources, nothing else.

Also, there are models trained specifically on valid medical info - PubMed literature, clinical notes, medical textbooks, and knowledge bases like UMLS, rather than general internet data .

That would not help in your example potentially, but would certainly negate the issue brought up by the OP.


Yes, you can ground the AI system to focus only on acceptable literature and not on what the model is trained on. However, I don't think most people would have access to these systems which need some work to build. (called Retrieval Augmented Generation Architecture) Most would just use ChatGPT or Copilot or something to get quick answers for free/cheap.
I am not aware if there are models that are just trained on Medical literature, that may be what some company needs to look into.

My point is that China and some other countries good at information warfare will use this to taint AI outputs with outcomes that they want. That will be the next frontier.

I thought this thread was all about the Wuhan Flu...then I noticed it was the usual InfinityAg Luddite post.

No medical practitioner is going to us a generic LLM for medical/clinical advice/treatments.

Like with anything AI related, domain specific AI's are what is driving ROI for AI and has been for 3 decades (yes, practical AI has been around a long time).

There are at least 5 domain specific medical AI's in use currently that I am aware of that have been built for specific purposes...for example, PathAI for pathology, AIDoc for radiology, AMBOSS for clinical use, MEDGemma for imaging, etc.

The issue with your use of only "acceptable literature" in LLMs, is who decides what is acceptable. If the answer is the Leftist Intelligentsia, then no thanks. Like with all automation, domain specific uses will provide a greater degree of accuracy and will find value in the marketplace.




I think I used the wrong word. I should have said "Approved", not "Acceptable".
Sorry.
reineraggie09
How long do you want to ignore this user?
AG
Agree you have people making the same mistakes as AI. But in AI driven health care, there are a lot fewer people to question the data at the practitioner level. Think of AI urgent cares, which already exist. There is "doctor" supervision but one doctor supervises several clinics. No way to monitor them all.

Also, that paper I referenced is on PubMed. A lot of crap on PubMed.
japantiger
How long do you want to ignore this user?
S
infinity ag said:

japantiger said:

infinity ag said:

Logos Stick said:

In your example though, if AI did not exist, you still have vets who are now refusing to prescribe those two meds together, which you've said is completely wrong.

You can always ground AI to prevent answers based on garbage, i.e. completely ignore this source when generating an answer, or use these sources and only those sources, nothing else.

Also, there are models trained specifically on valid medical info - PubMed literature, clinical notes, medical textbooks, and knowledge bases like UMLS, rather than general internet data .

That would not help in your example potentially, but would certainly negate the issue brought up by the OP.


Yes, you can ground the AI system to focus only on acceptable literature and not on what the model is trained on. However, I don't think most people would have access to these systems which need some work to build. (called Retrieval Augmented Generation Architecture) Most would just use ChatGPT or Copilot or something to get quick answers for free/cheap.
I am not aware if there are models that are just trained on Medical literature, that may be what some company needs to look into.

My point is that China and some other countries good at information warfare will use this to taint AI outputs with outcomes that they want. That will be the next frontier.

I thought this thread was all about the Wuhan Flu...then I noticed it was the usual InfinityAg Luddite post.

No medical practitioner is going to us a generic LLM for medical/clinical advice/treatments.

Like with anything AI related, domain specific AI's are what is driving ROI for AI and has been for 3 decades (yes, practical AI has been around a long time).

There are at least 5 domain specific medical AI's in use currently that I am aware of that have been built for specific purposes...for example, PathAI for pathology, AIDoc for radiology, AMBOSS for clinical use, MEDGemma for imaging, etc.

The issue with your use of only "acceptable literature" in LLMs, is who decides what is acceptable. If the answer is the Leftist Intelligentsia, then no thanks. Like with all automation, domain specific uses will provide a greater degree of accuracy and will find value in the marketplace.




I think I used the wrong word. I should have said "Approved", not "Acceptable".
Sorry.

If by Approved you mean what the current domain specific AIs use, then I think we agree...but again, this is not something that an LLM (non domain specific AI) lends itself to by nature. So not sure in a practical sense, how you land there unless, for example, an LLM simply outsources medical domain QnA to a medical domain AI.
Burdizzo
How long do you want to ignore this user?
AG
Cinco Ranch Aggie said:

Quote:

garbage in - garbage out

This has been a reality since computers came into existence.

AI is simply a fancy new implementation of an I/O device. If I was to train a model on volumes of data generated by the main stream media, I would be an obliviot to expect it to answer any questions with anything but MSM talking points and the like, since the MSM never reports on anything outside of their world view - or when they do, it is always in the negative.


One wrinkle to AI is that it uses statistics to determine the veracity of what it uses and what it generates. The Internet is saturated with data, some true some untrue. If more people believe something to be true then AI puts more weight on that when generating the answer. It may not matter if it is actually true, but AI will weight it as if it is true. Climate change is a good example. The more people that believe it to be true, the more AI will weight it as a fact even though science is scattered.

A wise politician will then flood the field with data sources showing their position makes more sense in order to shape AI responses, even though their position may be based on bunk. AI isn't the problem. Too much trust in AI is the problem
infinity ag
How long do you want to ignore this user?
Burdizzo said:

Cinco Ranch Aggie said:

Quote:

garbage in - garbage out

This has been a reality since computers came into existence.

AI is simply a fancy new implementation of an I/O device. If I was to train a model on volumes of data generated by the main stream media, I would be an obliviot to expect it to answer any questions with anything but MSM talking points and the like, since the MSM never reports on anything outside of their world view - or when they do, it is always in the negative.


One wrinkle to AI is that it uses statistics to determine the veracity of what it uses and what it generates. The Internet is saturated with data, some true some untrue. If more people believe something to be true then AI puts more weight on that when generating the answer. It may not matter if it is actually true, but AI will weight it as if it is true. Climate change is a good example. The more people that believe it to be true, the more AI will weight it as a fact even though science is scattered.

A wise politician will then flood the field with data sources showing their position makes more sense in order to shape AI responses, even though their position may be based on bunk. AI isn't the problem. Too much trust in AI is the problem


You got exactly what my point was.
I don't have a solution for this, but I can see this happening soon. Dems could flood the right data sources and anyone asking ChatGPT "Compare Newsom to Vance" will then get a skewed response making them vote for Newsom. A real problem.
Law-Apt_3G
How long do you want to ignore this user?
infinity ag said:

Burdizzo said:

Cinco Ranch Aggie said:

Quote:

garbage in - garbage out

This has been a reality since computers came into existence.

AI is simply a fancy new implementation of an I/O device. If I was to train a model on volumes of data generated by the main stream media, I would be an obliviot to expect it to answer any questions with anything but MSM talking points and the like, since the MSM never reports on anything outside of their world view - or when they do, it is always in the negative.


One wrinkle to AI is that it uses statistics to determine the veracity of what it uses and what it generates. The Internet is saturated with data, some true some untrue. If more people believe something to be true then AI puts more weight on that when generating the answer. It may not matter if it is actually true, but AI will weight it as if it is true. Climate change is a good example. The more people that believe it to be true, the more AI will weight it as a fact even though science is scattered.

A wise politician will then flood the field with data sources showing their position makes more sense in order to shape AI responses, even though their position may be based on bunk. AI isn't the problem. Too much trust in AI is the problem


You got exactly what my point was.
I don't have a solution for this, but I can see this happening soon. Dems could flood the right data sources and anyone asking ChatGPT "Compare Newsom to Vance" will then get a skewed response making them vote for Newsom. A real problem.


MSM skewed and already has patent on getting lemmings to the cliff.
agracer
How long do you want to ignore this user?
AG
Flagged for trolling
Refresh
Page 2 of 2
 
×
subscribe Verify your student status
See Subscription Benefits
Trial only available to users who have never subscribed or participated in a previous trial.