Born in Melbourne in 1930, he received a scholarship to study piano with Roy Shepherd at the Melbourne Conservatorium of Music. After obtaining diplomas in piano and composition, he moved to Paris. He studied piano with Blanche Bascouret at the École Normale de Musique. From 1957, his piano technique was recognized by René Leibowitz, a scholar of the Second Viennese School, and he served as an opera accompanist under Leibowitz, also receiving conducting lessons. He also participated in the activities of a group dedicated to performing 20th-century musical works, formed under conductor Diego Masson. From the 1960s, based at RCA in Rome, he began working in the world of arrangements and pop music, also starting activities in Spain and South America. In 1970, he returned to Paris, where he began composing music for films and documentary films.

In 1976, Whiffin decided to return to Australia. This was in response to an invitation from Professor Keith Humble, who had established the School of Music at La Trobe University in Melbourne. His work at the university included professorships in music theory and composition, as well as opportunities to perform his own works. While receiving composition commissions from numerous organizations, he also continued his career as a contemporary music pianist.

After returning to Australia, he also published a series of works titled "Contemporary Australian," introducing music for piano, violin, and flute.

After retiring as a principal professor at the university, he received the Australian Music Centre's Annual Contemporary Music Award in 1998 for "Murchitt," a work for nine instruments and poetry recitation, and the Albert Maggs Award in 2002 for his concerto for violin and five instruments.