A series of Fe-, Cr-, and Ni-doped TiO2 nanoparticles with doping concentration ranging from 2% to 10% are prepared by the sol-gel method for the study of their structural and magnetic properties. X-ray diffraction and energy dispersive x-ray analysis reveal that the particles are of a single anatase phase with successful substitution of Ti+4 with the dopant ions except Ni, which has a low solubility. Electron spin resonance (ESR) is performed with temperature variation to probe the magnetic interactions. For Fe-and Cr-doped nanoparticles, a similar temperature dependence of signal intensity is observed, indicating the same coupling mechanism. The major paramagnetic resonance signal at g similar to 2 suggests a dipolar interaction associated with the doping-induced free spins in the nanoparticle system whose strength increases with the concentration. This work demonstrates that ESR is a sensitive probe for distinguishing the type of spin interaction in dilute magnetic semiconductors.