Critical Point Theory for Indefinite Functionals with Symmetries

LetXbe a Hilbert space andφ∈C1(X, R) be strongly indefinite. Assume in addition that a compact Lie groupGacts orthogonally onXand thatφis invariant. In order to find critical points ofφwe develop the limit relative category of Fournieret al. in the equivariant context. We use this to prove two generalizations of the symmetric mountain pass theorem and a linking theorem. In the case of the mountain pass theorem the mountain range is allowed to lie in a subspace of infinite codimension. Also other conditions of the classical symmetric mountain pass theorem forG=Z/2 (due to Ambrosetti and Rabinowitz) can be weakened considerably. For example, we are able to deal with infinite-dimensional fixed point spaces. The proofs consist of a direct reduction to a relative Borsuk–Ulam type theorem. This provides a new proof even for the classical mountain pass theorem. The abstract critical point theorems are applied to an elliptic system with Dirichlet boundary conditions. We only need a weak version of the usual superquadraticity condition. The linking theorem can be applied to asymptotically linear Hamiltonian systems which are symmetric with respect to a (generalized) symplectic group action.