In considering a response that might have teeth it needs to be asked what is the most important aspect of the relationship. The cattle trade, students?
In truth there is not a lot Australia can do to cause Indonesia pain over the executions. There is little of importance in the relationship and that is a terrible condemnation of successive Australian governments. Both Labor and Liberal governments have thrown away leverage over stopping the boats, particularly Abbott and Morrison.
At the level of government in Indonesia there is little respect for Australia and the way it conducts itself, toward Indonesia, other Muslim countries and refugees. Australia is perceived as selfish, immature and racist.
Australia has demonstrated little understanding of the social, cultural and political complexities that are interwoven throughout the archipelago. Calling for a boycott of Bali falls into the category of immature.
Australia might withdraw its Ambassador and ask the Indonesians to do the same. It might restrict Indonesian travel to Australia, but such a move would be reciprocated. It could restrict Indonesian flights into Australian but again that would engender reciprocity.
One thing Australia might consider is to cease all contact and training with the Indonesian police and military for two years.