That is a question I have wondered many times myself.
I taught American lit and I always covered Brown when I talked about the Transcendentalists. Thoreau was a staunch defender of Brown and delivered speeches and wrote essays on his behalf.
Brown defended his actions by quoting from the Bible.
It is commonly accepted that Brown and some of his followers murdered some pro-slavery settlers in Kansas, but there was a lot of that going around.
I think Brown's position could be paraphrased as, You call me crazy, but we live in a society in which God's children are kept as work animals. What could be more crazy than that? They should be freed by any means necessary.
To me, Brown is similar to someone who is so opposed to abortion that he kills others in order to end it.
I found this excellent essay on Brown. It is supposedly a review of David Reynolds' massive biography on Brown (2005) but in fact becomes a long rumination on the meaning of Brown and his fixed obsession. Well worth a read.