The assailant had identified himself to police over the phone as Kenji Hayashi, a 62-year-old member of one of the largest crime syndicate groups in Japan, the Inagawa-kai.
The three wounded men were taken to hospital and one was in critical condition, Katsuzo Nakayama, a senior police spokesman, said.
Japan has strict gun-control laws and public shootings are rare.
Shootings that do occur are often linked to the Japanese mafia, yakuza, who have been linked to extortion, gambling, prostitution, gun-running, drug trafficking and construction-kickback schemes.
In 2007, the mayor of Nagasaki, Iccho Ito, was shot to death at close range by a gangster outside a train station while campaigning for re-election for his fourth term.
Later that year, a 37-year-old man killed two people and wounded six others in a shooting spree at a sports club in southern Sasebo city before killing himself outside a church.