You know a tube had gone bad when:
1) Your amp developes and uncontrollable high pitch squealing feedback. (a tube has gone microphonic. a microphonic tube can happen any time, but old age, or a sudden jar while still warm will be the most common causes)
2) Output power loss.
3) Loss of gain in a high gain amp. (can also be caused by a bad cable occasionally. check that first)
4) Tube color changes from warm orange to a blue, pr purple cast. Even if it still works fine, it's on it's last legs. Change it.
5) Amp used at the same settings is suddenly noisy with hiss, or hum.
6) Oscillating tonal changes.
7) Amp suddenly won't stay powered up, and you keep blowing the fuse.
8) When turned off, the tube is not clear but has a slight whitish cast.
These are the most common things I've experienced over the years using a tube amp.
Things to do when re-tubing:
1) If possible buy matched sets. Tubedepot.com offers to test the tubes for compatibility when you buy new ones.
2) When changing the power tubes also get the amp re-biased
3) If you have a Mesa, use Mesa tubes, and you don't need a re-bias.
4) Never touch the tube you are replacing bare handed. If you do, clean it with alcohol, let dry, and re-install it.
5) Never use anything metal around tube sockets to help pry it out. Not only can you shock the hell outta yourself even if it's not plugged in, you can damage the tube socket. The reason you can still get shocked with the amp unplugged is based on the use of a capacitor in most tube amps. It's basically a high voltage battery, and accidently arcing across the right set of pins will result in a good jolt.
6) Check tube compatibility. There are lots of replacement tubes that can be used like a Marshall uses an ECC83 preamp tube, but you can use a 12AX7 in place.
Tube life depends largely on use. I've seen pre-amp tubes last years, and seen them go in 8 months. High volumes, frequently moving gear, and overly high voltage at the wall are all factors in how long a tube set lasts.
It's never the whole set that goes, usually just one, but that's the indication it's time. With the exception of accidently breaking a tube while moving the amp.
Tube amp tones are heavily colored by the type of tubes in them. You can find tubes that brighten a dark amp, darken a bright amp, lower or raise gain, change power output, so on and on.
Here's a couple of links that are helpful.
At tube depot, if you click on the tube brand, and type, it gives you a brief overview of the tonal characteristics.
Hope any of this helps