If you are at the tip of Long Island, the best pastries and baked goods can be found at the Montauk Bake Shoppe. There may be a line but the jelly donuts are well worth the wait.
http://www.montaukbakeshoppe.com
29 The Plaza #A, Montauk, NY, 11954
(631) 668-2539