There is not a way to immediately delete a metric/tag, though they'll age out of the UI naturally after 24 hours.
With respect to monitors, the metric stream will stop being considered after that period.
With respect to dashboards, the metric/tag will still appear in the visualizations, but will not be available in the dropdown menus for graphing via our UI. The corresponding metric + tag space is however still available for graphing using the JSON method
If you're running the agent, assuming you've intentionally stopped or removed your host, all hosts or metrics that have not seen data in 24 hours will disappear from the UI, you may still query against them though it will not appear in drop downs or the infrastructure or host map views.