Try using the 'autotram' tool and click on the nearest end of the tunnel. Should work :)

The error message isn't super helpful for building tram on road tunnels and bridges (and vice versa). Aspects of the game UI can be fiddly eh :)

A problem that hardly exists, but it does. :)
In order to build or remove a railroad or road from an existing tunnel, you have to click on the appropriate and very small part of the tile. If the player clicks in another part of the field, the information about the need to destroy the tunnel will appear, which will be false.

This information makes some sense when a player wants to build a road or tracks over the entrance to the tunnel, but I don't think that is the correct and needed information.

In my opinion, any attempt to build a road or tracks on a tile with a tunnel, regardless of where you click, if the player is using an Autoroad/tram or road/tracks in the right direction, should end up building that road/tracks, or if the road/tracks already exist , a message about the existence of a road/tracks should appear.

In my case (OpenTTD 1.11.0), clicking on the lower part of a sloped tile will attempt to build a road dead-end on the lower part of said tile, which is compatible with a tunnel entrance (as it instead adds/removes road/tram from that tunnel). Clicking on the upper part of a sloped tile will attempt to build a road dead-end on the upper part of said tile. Since tunnel entrances can not have non-bridge roads above them, attempting to build a road on the upper part of said tile will fail with "Must demolish tunnel first".

In my case (OpenTTD 1.11.0), clicking on the lower part of a sloped tile will attempt to build a road dead-end on the lower part of said tile, which is compatible with a tunnel entrance (as it instead adds/removes road/tram from that tunnel). Clicking on the upper part of a sloped tile will attempt to build a road dead-end on the upper part of said tile. Since tunnel entrances can not have non-bridge roads above them, attempting to build a road on the upper part of said tile will fail with "Must demolish tunnel first".

Perfectly explained :)

That also means this is not really a bug, but the UX is not really superb. I myself too have issues when building tram-tracks that it is not always as clear to me what is going wrong.

Your mouse selects the whole tile, but for trams (and roads) the part you are on the tile is very important too. But this is not visually shown to the user, often leading to a bit frustrating results. For rails this is solved by showing what part of the rail will be build; which is especially useful for diagonal tracks.

So I wonder if we cannot do the same for tram-tracks, that we show a bit more where the tram-track will be build. That should also give feedback for instances like tunnel upgrading, the different between the entrance of the tunnel and the top of the tunnel.

Either way, I think this is a legit issue to fix, the UX part.