I don't think we showed it in the course. This is a setting on the tile itself in the tileset. To activate it:
Save and you can now jump through that tile!
It's weird because I already try that and it didn't work.
I then created a second tilemap with a new tileset and it did works in the end... I wonder what I did wrong with the first one...
It's easy to forget one setting, especially when learning a complex tool like Godot. Like when you code, sometimes a character or a variable name can break the logic.