Skip to main content
SpringerLink
Log in

Random Sampling with Removal

  • Ricky Pollack Memorial Issue
  • Published:
Discrete & Computational Geometry Aims and scope Submit manuscript

Abstract

We study randomized algorithms for constrained optimization, in abstract frameworks that include, in strictly increasing generality: convex programming; LP-type problems; violator spaces; and a setting we introduce, consistent spaces. Such algorithms typically involve a step of finding the optimal solution for a random sample of the constraints. They exploit the condition that, in finite dimension \(\delta \), this sample optimum violates a provably small expected fraction of the non-sampled constraints, with the fraction decreasing in the sample size r. We extend such algorithms by considering the technique of removal, where a fixed number k of constraints are removed from the sample according to a fixed rule, with the goal of improving the solution quality. This may have the effect of increasing the number of violated non-sampled constraints. We study this increase, and bound it in a variety of general settings. This work is motivated by, and extends, results on removal as proposed for chance-constrained optimization. For many relevant values of r, \(\delta \), and k, we prove matching upper and lower bounds for the expected number of constraints violated by a random sample, after the removal of k constraints. For a large range of values of k, the new upper bounds improve the previously best bounds for LP-type problems, which moreover had only been known in special cases, and not in the generality we consider. Moreover, we show that our results extend from finite to infinite spaces, for chance-constrained optimization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

Notes

  1. Note that both of these papers appeared in the same issue of the second volume of the Journal of Discrete & Computational Geometry.

References

  1. Adler, I., Shamir, R.: A randomized scheme for speeding up algorithms for linear and convex programming problems with high constraints-to-variables ratio. Math. Program. 61(1), 39–52 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  2. Amenta, N.: Helly-type theorems and generalized linear programming. Discrete Comput. Geom. 12(3), 241–261 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  3. Brise, Y., Gärtner, B.: Clarkson’s algorithm for violator spaces. Comput. Geom. 44(2), 70–81 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  4. Calafiore, G.C., Campi, M.C.: Uncertain convex programs: randomized solutions and confidence levels. Math. Program. 102(1), 25–46 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  5. Calafiore, G.C., Campi, M.C.: The scenario approach to robust control design. IEEE Trans. Autom. Control 51(5), 742–753 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  6. Campi, M.C., Garatti, S.: The exact feasibility of randomized solutions of uncertain convex programs. SIAM J. Optim. 19(3), 1211–1230 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  7. Campi, M.C., Garatti, S.: A sampling-and-discarding approach to chance-constrained optimization: feasibility and optimality. J. Optim. Theory Appl. 148(2), 257–280 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  8. Clarkson, K.L.: New applications of random sampling in computational geometry. Discrete Comput. Geom. 2(2), 195–222 (1987)

    Article  MathSciNet  MATH  Google Scholar 

  9. Clarkson, K.L.: A Las Vegas algorithm for linear programming when the dimension is small. In: Proceedings of the 29th Annual IEEE Symposium on Foundations of Computer Science, pp. 452–456. IEEE (1988)

  10. Clarkson, K.L.: A bound on local minima of arrangements that implies the upper bound theorem. Discrete Comput. Geom. 10(4), 427–433 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  11. Clarkson, K.L.: Las Vegas algorithms for linear and integer programming when the dimension is small. J. Assoc. Comput. Mach. 42(2), 488–499 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  12. Clarkson, K.L., Mehlhorn, K., Seidel, R.: Four results on randomized incremental constructions. Comput. Geom. 3(4), 185–212 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  13. Clarkson, K.L., Shor, P.W.: Applications of random sampling in computational geometry II. Discrete Comput. Geom. 4(5), 387–421 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  14. Cormen, T.H., Leiserson, Ch.E., Rivest, R.L.: Introduction to Algorithms. MIT Electrical Engineering and Computer Science Series. MIT Press, Cambridge (1990)

  15. De Loera, J.A., La Haye, R.N., Oliveros, D., Roldán-Pensado, E.: Chance-constrained convex mixed-integer optimization and beyond: two sampling algorithms within \(S\)-optimization. J. Convex Anal. 25(1), 201–218 (2018)

    MathSciNet  MATH  Google Scholar 

  16. Dyer, M.E., Frieze, A.M.: A randomized algorithm for fixed-dimensional linear programming. Math. Program. 44(2), 203–212 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  17. Fagiano, L., Schildbach, G.: Sampling lemma with a-posteriori sample removal (2013). Unpublished mansuscript

  18. Fischer, K., Gärtner, B.: The smallest enclosing ball of balls: combinatorial structure and algorithms. Int. J. Comput. Geom. Appl. 14(4–5), 341–378 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  19. Gärtner, B.: A subexponential algorithm for abstract optimization problems. In: Proceedings of the 33rd Annual IEEE Symposium on Foundations of Computer Science, pp. 464–472. IEEE (1992)

  20. Gärtner, B.: Sampling with removal in LP-type problems. J. Comput. Geom. 6(2), 93–112 (2015)

    MathSciNet  MATH  Google Scholar 

  21. Gärtner, B., Matoušek, J., Rüst, L., Škovroň, P.: Violator spaces: structure and algorithms. Discrete Appl. Math. 156(11), 2124–2141 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  22. Gärtner, B., Schurr, I.: Linear programming and unique sink orientations. In: Proceedings of the 17th Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 749–757. ACM, New York (2006)

  23. Gärtner, B., Welzl, E.: Linear programming – randomization and abstract frameworks. Proceedings of the 13th Annual Symposium on Theoretical Aspects of Computer Science (Grenoble, 1996). Lecture Notes in Computer Science, vol. 1046, pp. 667–687. Springer, Berlin (1996)

  24. Gärtner, B., Welzl, E.: A simple sampling lemma: analysis and applications in geometric optimization. Discrete Comput. Geom. 25(4), 569–590 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  25. Haussler, D., Welzl, E.: \(\epsilon \)-Nets and simplex range queries. Discrete Comput. Geom. 2(2), 127–151 (1987)

    MathSciNet  MATH  Google Scholar 

  26. Hoeffding, W.: Probability inequalities for sums of bounded random variables. J. Am. Statist. Assoc. 58, 13–30 (1963)

    Article  MathSciNet  MATH  Google Scholar 

  27. Kalai, G.: A subexponential randomized simplex algorithm. In: Proceedings of the 24th Annual ACM Symposium on Theory of Computing, pp. 475–482 (1992)

  28. Matoušek, J.: Removing degeneracy in LP-type problems revisited. Discrete Comput. Geom. 42(4), 517–526 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  29. Matoušek, J., Sharir, M., Welzl, E.: A subexponential bound for linear programming. Algorithmica 16(4–5), 498–516 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  30. Megiddo, N.: Linear programming in linear time when the dimension is fixed. J. Assoc. Comput. Mach. 31(1), 114–127 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  31. Mustafa, N.H., Varadarajan, K.: Epsilon-approximations & epsilon-nets. In: Goodman, J.E., O’Rourke, J., Tóth, C.D. (eds.) Handbook of Discrete and Computational Geometry, chapter 47. Discrete Mathematics and its Applications. CRC Press, Boca Raton (2018)

    Google Scholar 

  32. Scarf, H.E.: An observation on the structure of production sets with indivisibilities. Proc. Nat. Acad. Sci. USA 74(9), 3637–3641 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  33. Seidel, R.: Small-dimensional linear programming and convex hulls made easy. Discrete Comput. Geom. 6(5), 423–434 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  34. Sharir, M., Welzl, E.: A combinatorial bound for linear programming and related problems. In: Proceedings of the 9th Annual Symposium on Theoretical Aspects of Computer Science (Cachan, 1992), pp. 569–579, Lecture Notes in Computer Science, vol. 577. Springer, Berlin (1992)

  35. Škovroň, P.: Abstract Models of Optimization Problems. PhD thesis, Charles University, Prague (2007). https://is.cuni.cz/webapps/zzp/detail/43007/?lang=en

  36. Szedlák, M.: Redundancy in Linear Systems: Combinatorics, Algorithms and Analysis. PhD thesis, ETH Zürich, Zürich (2017). https://doi.org/10.3929/ethz-b-000167108

  37. Welzl, E.: Smallest enclosing disks (balls and ellipsoids). In: New Results and New Trends in Computer Science (Graz, 1991). Lecture Notes in Comput. Sci., vol. 555, pp. 359–370. Springer, Berlin (1991)

Download references

Acknowledgements

The authors are grateful to Chih-Hung Liu and Emo Welzl for sharing important insights. Furthermore we thank Luis Barba for helpful discussions.

Author information

Authors and Affiliations

  1. IBM Research, 650 Harry Road, San Jose, CA, 95120, USA

    Kenneth L. Clarkson

  2. Department of Computer Science, Institute of Theoretical Computer Science, ETH Zürich, 8092, Zürich, Switzerland

    Bernd Gärtner, Johannes Lengler & May Szedlák

Authors
  1. Kenneth L. Clarkson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernd Gärtner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Johannes Lengler
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. May Szedlák
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bernd Gärtner.

Additional information

Editor in Charge: János Pach

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Dedicated to the memory of Ricky Pollack.

The research of the last author was supported by the Swiss National Science Foundation (SNF Project 200021_150055/1).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Clarkson, K.L., Gärtner, B., Lengler, J. et al. Random Sampling with Removal. Discrete Comput Geom 64, 700–733 (2020). https://doi.org/10.1007/s00454-020-00193-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00454-020-00193-0

Keywords

Search

Navigation