In this regard, the Wikipedia article on pepper spray is interesting. I see that it has long been controversial in the States for its likely contribution to scores of deaths, and on the international scene, it is has very variable regulation. It's not at all uncommon for it to be banned entirely, while other countries may allow it under licence, or for use only in protection against animals.
Tasers are of course controversial for their potential lethality too. Wikipedia indicates that they are illegal for the public to have just about everywhere, except the Czech republic, and of course, many parts of the United States.
As for knives: well, they can do a pretty good job at mass killing too, and I am not surprised that they are regulated and that police have concerns about certain groups having one in their possession. I'm not entirely sure how one regulates so that the police can take one off a bunch of drunken youths in a nightclub area, but leave it with the young woman coming home from the office who thinks it will be useful in self defence.
And really, the dubious utility of allowing people to arm themselves is the big problem for all self defence. First, the chances of involvement with violent crime for most people, in the course of a lifetime, in a country like Australia, is very very small. Worrying about being armed against attack in a normal day is, I would say, a touch paranoid for nearly all men. (It's less so for women who are out at night, unfortunately, but statistically I would be sure the reality is far different from the perception.) But for those who do have exposure to danger, there is no certainty at all that having a non lethal form of self defence is going to be accessible or useful in the event of attack.
And, of course, the number of cases in which self defence items are successfully deployed has to be considered in light of the number of times criminals may successful use them aggressively for their own purposes. And that certainly happens with non lethal items as well as with guns - see these articles from the States in 1995 and just this year about the criminal use of pepper spray, for example.
In the big picture of what's better for society overall, I think most Australians are comfortable with what's illegal from a potential weapon point of view.
Update: even nice old Canada can have criminal problems with pepper spray - where it appears popular for protection against bears - as appears from this report from earlier this year. There are some surprising figures:
CALGARY – Police say there’s an alarming increase in the use of pepper spray by local criminals.
In 2011, police recorded 88 incidents where pepper spray was used. A year later that number almost doubled to 161. Then, in 2013 there were 147 incident in the first nine months, which suggests an upward trend
The latest pepper spray incident was during a robbery at the Bay location at Market Mall on Wednesday.