Nominally the ADV7182 is designed to work with NTSC signals divided down by the input 24 / 51 ohm resistor network. You loose the differential benefits when you try to do what you are doing. Also your selection of divider network has changed the RC time constants of the input when you use the values you shown.
It sounds like the differential signal is 1Vpp input to your circuit that is 0.5Vpp single ended which you further divide down byt the 1.3k / 430 resistor network giving you much less input the the chip the 0.5V. Is this correct?