Nuwanidu Fernando was the top scorer for Sri Lanka with a 63-ball 50, while Mendis and Dunith Velez scored 34 and 32 respectively.
India quickly lost the wicket of captain Rohit Sharma (17) in their reply, while Virat Kohli, who scored a century in the first match, was dismissed for four runs.
Rahul and Hardik Pandya (36) put on 75 runs for the fifth wicket to take India closer to their target, with Rahul sticking it out till the end to take them to the finish line.
The third and final ODI will be played in Thiruvananthapuram on Sunday.