Inclusion Criteria (Phase I): - Histologically or cytologically confirmed epithelial ovarian cancer, primary peritoneal serous cancer, fallopian tube cancer, or triple-negative breast cancer - Ovarian, primary peritoneal, and fallopian tube participants must have either measurable disease by RECIST criteria or an elevated CA125 level at least twice the upper limit of normal on two separate occasions at least 1 day but not more then 3 months apart. Breast cancer participants must have measurable disease by RECIST criteria. - Prior chemotherapy for ovarian cancer patients must have included a first-line platinum-based regimen with or without intravenous consolidation chemotherapy - Breast cancer patients must have recurred post both an adriamycin-and taxane-containing regimen and have had at least 1 chemotherapy for metastatic breast cancer - Prior hormonal-based therapy for ovarian, primary peritoneal serous, fallopian tube cancer, or breast cancer is acceptable. - Patients may not have previously received a PARP-inhibitor - Patients may not have had a prior anti-angiogenic agent in the recurrent setting - 18 years of age or older - Life expectancy of greater than 6 months - ECOG performance status 0 or 1 - Adequate organ and marrow function as outlined in the protocol - Toxicities of prior therapy (excepting alopecia) should be resolved to less than or equal to Grade 1 as per NCI-CTCAE active version - Subjects with limited stage basal cell or squamous cell carcinoma of the skin or carcinoma in situ of the breast or cervix are eligible. Subjects with stage I or II cancer treated with a curative intent with no evidence of recurrent disease 5 years following diagnosis are eligible - Women of child-bearing potential and men must agree to use adequate contraception prior to study entry, for the duration of the study participation, and for 3 months following treatment discontinuation - Must be able to tolerate oral medications and not have gastrointestinal illnesses that would preclude absorption of cediranib or olaparib - Patients must be willing and able to check and record daily blood pressure readings Additional inclusion criteria (Phase II only): - Participants must have histologically or cytologically confirmed epithelial ovarian cancer, primary peritoneal serous cancer, or fallopian tube cancer - May have received up to 1 non-platinum-based line of therapy in the recurrent setting - Participants should have platinum-sensitive disease, where platinum-sensitive is defined as having had a >6 month interval since last receiving platinum therapy prior to disease recurrence. Patients must have had a prior response whole on the platinum-containing regimen and cannot have experienced disease progression while receiving platinum Exclusion Criteria: - Participants who have had chemotherapy or radiotherapy within 3 weeks (6 week for nitrosoureas or mitomycin C) prior to entering the study or those who have not recovered from adverse events due to agent administered more than 3 weeks earlier - Participants may not be receiving any other investigational agents nor have participated in an investigational trial within the past 4 weeks. Subjects may not have received prior treatment affecting the VEGF pathway in the recurrent setting, including thalidomide, bevacizumab, sunitinib, or sorafenib. Subjects may not have received prior treatment with oregovomab (OvaRex) or any other antibodies that may interfere with CA-125 measurements - Patients with untreated brain metastases, spinal cord compression, or evidence of symptomatic brain metastases or leptomeningeal disease as noted on screening CT or MRI scans should not be included on this study since neurologic dysfunction may confound the evaluation of neurologic and other adverse events - History of allergic reactions attributed to compounds of similar chemical or biologic composition to cediranib or olaparib - Participants receiving any medications or substances that are inhibitors or inducers of CYP3A4 or CYP1A2 are ineligible - Patients with any of the following: 1)History of myocardial infarction within six months, 2) QTc prolongation > 500 msec or other significant ECG abnormality noted within 14 days of treatment, 3) NYHA classification of III or IV, 4) Condition requiring use of drugs or biologics with proarrhythmic potential - History of stroke or transient ischemic attack within six months - Inadequately controlled hypertension on antihypertensive medications - Any prior history of hypertensive crisis or hypertensive encephalopathy - Clinically significant peripheral vascular disease or vascular disease - Unstable angina - Major surgical procedure, open biopsy, or significant traumatic injury within 28 days prior to starting cediranib - History of abdominal fistula, gastrointestinal perforation or intra-abdominal abscess - Current signs and/or symptoms of bowel obstruction - Current dependency on IV hydration or TPN - Evidence of coagulopathy or bleeding diathesis - Uncontrolled intercurrent illness - Known HIV-positive individuals - Pregnant women - Patients with pre-existing grade 2 or higher peripheral neuropathy