to remove duplicates within file, when duplicates are not one after another (adjacent duplicate lines), and preserver order so that last duplicate line is last in file

awk '!visited[$0]++' YOUR_FILE_HERE > YOUR_FILE_HERE.temp && mv YOUR_FILE_HERE.temp YOUR_FILE_HERE