The Channel Islands and the Isle of Man are Crown Dependencies of the British Crown and as such are not  part of the EU.

Their relationship with the EU is governed principally by Protocol 3 to the UK Treaty of Accession  to the European Community of 1972 (as re-confirmed in subsequent EU treaties), which made these territories effectively part of the EU Customs Union and Common Customs Tariff and, for the purposes of the import and export of goods, also part of the EU Single Market in certain respects.

Whilst the Isle of Man is not part of the UK, it is treated as if it were part of the UK for VAT purposes. The Channel Islands is neither part of the UK nor treated as if it were part of the UK for VAT purposes.

If Brexit occurs, it is likely that that the legal relationship of the Channel Islands and the Isle of Man with the EU will change in the same sort of way as will the UK relationship with the EU. This means that, in the event of a "no deal" Brexit, the Protocol 3 arrangements of the Channel Islands and the Isle of Man with the EU will cease entirely and immediately and the British Crown (in practice acting in close alignment with the UK Government and the Jersey, Guernsey and Isle of Man Administrations) will be left to protect the external interests of  these Crown Dependency Territories as regards the EU.  The Isle of Man will continue to be aligned with the UK for VAT purposes but not so, the Channel Islands  - unless and until in each case the VAT tax is changed and/or modified.

In the event of a "deal" proceeding in line with the  current version of the EU-UK Withdrawal Agreement of 17th October 2019, these  Crown Dependencies would (to the extent that they had been integrated into the EU framework by Protocol 3 in the first place) have the benefit of the transitional implementation period up to and including 31st December 2020  in common with the UK and would then need to rely on arrangements negotiated by the British Crown (acting in close alignment with the UK etc as above-mentioned) thereafter. 

Whilst some arrangements have been put in place by the local Administrations to deal with essential stockpiling of medicines and vital foodstuffs in the event of a "no deal" Brexit, there does not appear to be any overt  sense of widespread concern as to what the future holds for the Channel Islands or the Isle of Man thereafter. That is not to say that complacency has set in.

The mood at the moment seems to be the familiar and rather comforting British approach of "Keep calm and carry on!" 

These territories, particularly in the context of their highly developed financial services sectors, are used to dealing with all parts of the World and that is perhaps why Brexit may not seem as much of a challenge there as opposed to the UK itself, whose economy in many ways is highly integrated with the economies of EU member states.

What happens in the coming period  will be interesting!