Backport:

core: Fix breakage of Listen directives with MPMs that use a
per-directory config. PR 52904.
Submitted by: sf
Reviewed by: covener, minfrin


git-svn-id: https://svn.apache.org/repos/asf/httpd/httpd/branches/2.4.x@1305145 13f79535-47bb-0310-9956-ffa450edef68