Only flag line as modifed if no source is selected.
Don't flag lines removed from one source as modified when selecting that source.