This phase I trial studies the side effects and best dose of ziv-aflibercept when given together with pembrolizumab in treating patients with solid tumors that have spread to other places in the body. Ziv-afibercept works by decreasing blood and nutrient supply to the tumor, which may result in shrinking the tumor. Monoclonal antibodies, such as pembrolizumab, may block tumor growth in different ways by targeting certain cells. Giving ziv-aflibercept together with pembrolizumab may be a better treatment for patients with advanced solid tumors.