They aren't making light of what you said. But that is literally the definition of hearsay:
hearsay
/hirs/
Learn to pronounce
noun
information received from other people that one cannot adequately substantiate; rumor.
"according to hearsay, Bob had managed to break his arm"
LAW
the report of another person's words by a witness, which is usually disallowed as evidence in a court of law.
"everything they had told him would have been ruled out as hearsay"
hearsay
/hirs/
Learn to pronounce
noun
information received from other people that one cannot adequately substantiate; rumor.
"according to hearsay, Bob had managed to break his arm"
LAW
the report of another person's words by a witness, which is usually disallowed as evidence in a court of law.
"everything they had told him would have been ruled out as hearsay"