5. Projects in Support Phase » New Help Desk Project » #193 - Re: Center Stage Wines Redirect Issue

Everyone,

To add to the above, the Apache setup was really messy. Here's what we
found:

- The Options directive was set up incorrectly
- There were issues with the VirtualHosts causing "client denied by
server configuration"
- There were settings in httpd.conf that belonged in VirtualHosts

One problem that makes it hard to find environment-specific issues is that
normally we can be pretty hands-off with server configuration. The problem
with that is we have to make an assumption that all server changes are
being tracked somewhere, and unfortunately they're not. Sometimes Rackspace
will leave a bunch of backup files in the live directory of config files,
but 1) they don't follow a naming convention and 2) they just shouldn't be
there.

I've asked Rackspace that in the future all changes are documented. What
I'm proposing to you, that we have done for other clients is that I can
create a Git repository that holds all server configuration files that we
rely on - Apache, PHP, etc. Whenever anyone makes changes, those changes
should be committed to the Git repository (or sent to us so we can).

One thing that I'm not a fan of with live server configuration is guessing.
Rackspace does a lot of this - "Let's increase this setting a little to see
if it helps". We prefer a formulaic approach, and based on our history with
you (La Grange), I know that is something that you appreciate as well.

Please let me know what you think about this. It may take time to figure
out what is different about this environment, but thanks to the flexibility
of Magento we could use a different approach (a very small changeset) to
work around it.

Thanks,
Bob


On Sat, Jun 14, 2014 at 9:14 AM, Robert Brodie wrote:

> Everyone,
>
> After a few Apache fixes and a workaround in the code, we were able to get
> rid of the redirect. I believe there may be something abnormal with the
> network setup for the following reasons:
>
> 1. The redirect didn't happen until the separate admin server was put
> in place
> 2. We created a similar setup locally (different admin URL) and the
> redirect would not happen
> 3. There were Apache setup issues
>
> Under normal circumstances, Magento allows us to call other controller
> indexes whether the admin is on the same server or not. We had to change
> this a bit and go a different route. I would like to find out why we had to
> implement an environment-specific change, but right now it is working
> without the redirect. We found nothing that was calling a backend route or
> model, and this didn't happen in other environments.
>
> Thanks,
> Bob
>
>
> On Sat, Jun 14, 2014 at 12:19 AM, Robert Shires > robert@lagrangesystems.com> wrote:
>
>> Hi Bob,
>>
>> Our standard setup for DNS is to have the root domain (
>> centerstagewines.com) point to our redirect servers:
>>
>> centerstagewines.com. 3290 IN A 107.21.213.247
>> centerstagewines.com. 3290 IN A 54.214.232.160
>> centerstagewines.com. 3290 IN A 107.21.212.122
>>
>> Theses servers sole responsibility is to redirect centerstagewines.com
>> to www.centerstagewines.com. We do this because you cannot have a CNAME
>> assigned to a root level domain. A root level domain must have IP address
>> only. You can only have a CNAME on a third level domain like
>> www.centerstagewines.com.
>>
>> We point www.centerstagewines.com to the CNAME
>> pxlqtij.lagrangesystems.net because we can manage the IP addresses of
>> this CNAME. Lagrange Systems balancers will add or remove IP address as
>> needed to manage load and performance.
>>
>> Currently this CNAME points to the following IP address:
>> jpxlqtij.lagrangesystems.net. 59 IN A 23.253.107.46
>> jpxlqtij.lagrangesystems.net. 59 IN A 162.242.226.124
>> jpxlqtij.lagrangesystems.net. 59 IN A 162.242.235.243
>>
>> These IP addresses may change from time to time depending on many
>> factors. We do not redirect any paths like we are seeing with the
>> redirection from www.centerstagewines.com to
>> https://admin.centerstagewines.com/index.php/. We also do not redirect
>> from HTTP to HTTPS.
>>
>> We can bypass our service completely by explicitly setting
>> www.centerstagewines.com to point to 72.32.214.98 in our hosts file and
>> we still see the redirect to
>> https://admin.centerstagewines.com/index.php/.
>>
>> We currently don't have Super User access to the production server or the
>> DB server to help in any investigation. We are here to help in any way.
>>
>> Robert Shires
>> 303-359-8383
>> Lagrangesystems.com
>>
>>
>> On Fri, Jun 13, 2014 at 3:23 PM, Robert Brodie
>> wrote:
>>
>>> Hi Mike and John G.,
>>>
>>> I ran dig on centerstagewines.com and while it does redirect, that's
>>> not the ideal setup. Using dig, I found that it's not going through the
>>> broker. Since it's doing that redirect, the WEB1 server is unaware that
>>> itself is centerstagewines.com.
>>>
>>> If you look at Apache, the virtual hosts are set up with ServerName
>>> centerstagewines.com with a ServerAlias of www.centerstagewines.com.
>>> Since Apache expects the domain to be associated with that IP address,
>>> there could potentially be issues. There shouldn't, but there can be.
>>>
>>> To rule out network issues, please be sure that centerstagewines.com
>>> and www.centerstagewines.com are set up the same. Here are the results
>>> of the two dig commands:
>>>
>>> *dig -t ANY centerstagewines.com *
>>> ;; Truncated, retrying in TCP mode.
>>>
>>> ; > DiG 9.8.3-P1 > -t ANY centerstagewines.com
>>> ;; global options: +cmd
>>> ;; Got answer:
>>> ;; ->>HEADER>>> ;; flags: qr rd ra; QUERY: 1, ANSWER: 14, AUTHORITY: 0, ADDITIONAL: 0
>>>
>>> ;; QUESTION SECTION:
>>> ;centerstagewines.com. IN ANY
>>>
>>> ;; ANSWER SECTION:
>>> centerstagewines.com. 3599 IN SOA ns73.domaincontrol.com. dns.jomax.net.
>>> 2014060802 28800 7200 604800 3600
>>> centerstagewines.com. 3599 IN A 54.214.232.160
>>> centerstagewines.com. 3599 IN A 107.21.213.247
>>> centerstagewines.com. 3599 IN A 107.21.212.122
>>> centerstagewines.com. 3599 IN MX 10 ASPMX.L.GOOGLE.com.
>>> centerstagewines.com. 3599 IN MX 20 ALT1.ASPMX.L.GOOGLE.com.
>>> centerstagewines.com. 3599 IN MX 30 ALT2.ASPMX.L.GOOGLE.com.
>>> centerstagewines.com. 3599 IN MX 40 ASPMX2.GOOGLEMAIL.com.
>>> centerstagewines.com. 3599 IN MX 50 ASPMX3.GOOGLEMAIL.com.
>>> centerstagewines.com. 3599 IN NS ns73.domaincontrol.com.
>>> centerstagewines.com. 3599 IN NS ns74.domaincontrol.com.
>>> centerstagewines.com. 3599 IN TXT "v=spf1 a mx include:spf.mtasv.net
>>> ~all"
>>> centerstagewines.com. 3599 IN TXT "v=spf1 mx ptr ptr:174.143.14.247
>>> include:secureserver.net -all"
>>> centerstagewines.com. 3599 IN TXT
>>> "google-site-verification=wqDlxasxUPUMhO3iV_9ag_OEzezwkXScgWnJDlv9JaY"
>>>
>>> ;; Query time: 88 msec
>>> ;; SERVER: 8.8.8.8#53(8.8.8.8)
>>> ;; WHEN: Fri Jun 13 17:15:42 2014
>>> ;; MSG SIZE rcvd: 524
>>>
>>> *dig -t ANY www.centerstagewines.com *
>>> ; > DiG 9.8.3-P1 > -t ANY www.centerstagewines.com
>>> ;; global options: +cmd
>>> ;; Got answer:
>>> ;; ->>HEADER>>> ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
>>>
>>> ;; QUESTION SECTION:
>>> ;www.centerstagewines.com. IN ANY
>>>
>>> ;; ANSWER SECTION:
>>> www.centerstagewines.com. 3599 IN CNAME jpxlqtij.lagrangesystems.net.
>>>
>>> ;; Query time: 88 msec
>>> ;; SERVER: 8.8.8.8#53(8.8.8.8)
>>> ;; WHEN: Fri Jun 13 17:15:50 2014
>>> ;; MSG SIZE rcvd: 84
>>>
>>> Thanks,
>>> Bob
>>>
>>>
>>> On Fri, Jun 13, 2014 at 5:18 PM, Mike Haller
>>> wrote:
>>>
>>>> Hi Bob,
>>>>
>>>> centerstagewines.com gets you a redirect to www.centerstagewines.com.
>>>> The site http://www.centerstagewines.com should work by itself
>>>> shouldn't it? When I was checking the web1 server, I tried to do a curl
>>>> request from itself to its local IP address, giving
>>>> centerstagewines.com in the host header. That properly redirected me
>>>> to http://www.centerstagewines.com. Requesting that URL locally
>>>> resulted in the redirect to the admin server. This is without involving
>>>> anything other than the application server itself, so it seems the problem
>>>> is confined to the server. Checking the apache configuration didn't reveal
>>>> any redirect there, and sibling resources to index.php did not redirect.
>>>> This leads me to conclude that there is something in Magento that must be
>>>> doing it.
>>>>
>>>> Mike
>>>>
>>>>
>>>> On Jun 13, 2014, at 15:01, Robert Brodie wrote:
>>>>
>>>> La Grange, is there someone we can call to get this sorted out?
>>>>
>>>>
>>>> On Fri, Jun 13, 2014 at 4:30 PM, Robert Brodie
>>>> wrote:
>>>>
>>>>> Hi John,
>>>>>
>>>>> Sean and I have been digging into it, but I'm still seeing network
>>>>> issues. I'm seeing that http://centerstagewines.com still points
>>>>> to 107.21.212.122. What is that server? I'd really like to see if pointing
>>>>> centerstagewines.com to the same IP as www.centerstagewines.com
>>>>> produces the same issue.
>>>>>
>>>>> We've gone through our controllers, and they're set up as they
>>>>> normally are for other projects, and aren't calling any redirects or admin
>>>>> routes. With that, we've set up our local environments to have separate
>>>>> admin URLs and aren't reproducing the issue. We'll continue to look today
>>>>> and through the weekend. If La Grange could point the non-www domain to the
>>>>> same IP as the www subdomain, that would be greatly appreciated.
>>>>>
>>>>> Thanks,
>>>>> Bob
>>>>>
>>>>>
>>>>> On Fri, Jun 13, 2014 at 3:22 PM, John Godish wrote:
>>>>>
>>>>>> Any progress?
>>>>>> _______________________
>>>>>>
>>>>>>
>>>>>> *John *
>>>>>> On 6/13/2014 12:11 PM, Robert Brodie wrote:
>>>>>>
>>>>>> Not a version issue, no. There may be an issue with how it's
>>>>>> interpreting a route on one of the controller methods. Usually separating
>>>>>> the admin doesn't cause this (we have one with a bunch of stores), but it's
>>>>>> always possible. I'll keep looking into Apache while the guys are looking
>>>>>> at the internals. What's interesting is that it loads the entire page, and
>>>>>> then redirects.
>>>>>>
>>>>>>
>>>>>> On Jun 13, 2014, at 12:04 PM, John Godish wrote:
>>>>>>
>>>>>> Not a Magento version issue, correct?
>>>>>> _______________________
>>>>>> *John Godish*
>>>>>> Managing Director - IT
>>>>>> Ashburn Corp
>>>>>> D.B.A. WinesTilSoldOut.com
>>>>>> CenterStageWines.com
>>>>>> Roger Wilco
>>>>>> Ph: 856-356-2524 x312
>>>>>> Johng@WTSO.com
>>>>>> On 6/13/2014 11:59 AM, Robert Brodie wrote:
>>>>>>
>>>>>> Thanks, Jay. I think we have found something and are looking at it.
>>>>>> We've never seen this particular issue so it could be a bug in Magento we
>>>>>> need to work around (it doesn't happen on Treehouse for example).
>>>>>>
>>>>>> Also, I've cleaned up the virtual hosts on WEB1. There were a few
>>>>>> things out of place and it was throwing a "client denied by configuration".
>>>>>>
>>>>>> Thanks,
>>>>>> Bob
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jun 13, 2014, at 11:00 AM, James Smith
>>>>>> wrote:
>>>>>>
>>>>>> Hi Everyone,
>>>>>>
>>>>>> The DNS entry for centerstagwines.com is correct. That is our
>>>>>> redirect service that remaps centerstagewines.com to
>>>>>> www.centerstagewines.com. This redirection is working properly.
>>>>>> Also, using nslookup as opposed to ping you will see that there are three
>>>>>> servers of ours handling centerstagewines.com as well as the three
>>>>>> servers handling www.centerstagewines.com.
>>>>>>
>>>>>> Thanks,
>>>>>> Jay
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>> On Jun 13, 2014, at 9:56 AM, Robert Brodie
>>>>>> wrote:
>>>>>>
>>>>>> Hi Everyone,
>>>>>>
>>>>>> I did find one issue with the Apache setup - it didn't have the
>>>>>> Indexes option set. That didn't fix the issue, but it did remove the
>>>>>> 'Client denied by configuration" issue.
>>>>>>
>>>>>> Additionally, I noticed this:
>>>>>> [robert@Brodie-iMac:~]$ ping www.centerstagewines.com
>>>>>> PING jpxlqtij.lagrangesystems.net (162.242.235.243): 56 data bytes
>>>>>> 64 bytes from 162.242.235.243: icmp_seq=0 ttl=52 time=8.791 ms
>>>>>>
>>>>>> [robert@Brodie-iMac:~]$ ping centerstagewines.com
>>>>>> PING centerstagewines.com (54.214.232.160): 56 data bytes
>>>>>> Request timeout for icmp_seq 0
>>>>>>
>>>>>> Shouldn't centerstagewines.com be pointed to the 243 address as
>>>>>> well?
>>>>>>
>>>>>> Thanks,
>>>>>> Bob
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 12, 2014 at 8:10 PM, Robert Brodie
>>>>>> wrote:
>>>>>>
>>>>>>> Jay,
>>>>>>>
>>>>>>> Could you help us compare our Apache setup against another client
>>>>>>> to see if we can spot any issues? I'll send the credentials separately.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Bob
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jun 12, 2014 at 5:02 PM, Robert Brodie >>>>>>> > wrote:
>>>>>>>
>>>>>>>> Hey guys,
>>>>>>>>
>>>>>>>> *Here's what's going on:*
>>>>>>>> When you go to http://www.centerstagewines.com, you're redirected
>>>>>>>> to https://admin.centerstagewines.com/index.php. That's strange in
>>>>>>>> itself. Then, since it wasn't accessed via the admin path
>>>>>>>> (index.php/admin), it redirects to the homepage, maintaining the protocol
>>>>>>>> https, landing you on https://www.centerstagewines.com.
>>>>>>>>
>>>>>>>> I called back and the St. Account Manager that updated the ticket
>>>>>>>> wasn't available, but another tech (Bob) is looking into it.
>>>>>>>>
>>>>>>>> *What We've Done*
>>>>>>>>
>>>>>>>> 1. (SUMO) I've looked over the Magento config and I'm not
>>>>>>>> seeing anything out of place. The normal unsecure and secure URLs are set,
>>>>>>>> as well as the custom admin URL. Jay, this is similar to Treehouse but with
>>>>>>>> a normal /admin path.
>>>>>>>> 2. (LG) Jay has make sure that we're pointing to the correct
>>>>>>>> servers
>>>>>>>>
>>>>>>>> *Things To Do*
>>>>>>>>
>>>>>>>> 1. Bob at RS is looking over the Apache configuration to sort
>>>>>>>> out the redirect
>>>>>>>>
>>>>>>>> *Notes*
>>>>>>>> The initial load (before redirect) is showing a load time of less
>>>>>>>> than 200ms on average. The two redirects take that time up to almost 2s,
>>>>>>>> and that time increases exponentially as more traffic is added. Once the
>>>>>>>> redirect is addressed, we should see times closer to the 200ms.
>>>>>>>>
>>>>>>>> Thanks again,
>>>>>>>> Bob
>>>>>>>>
>>>>>>>> --
>>>>>>>> Robert Brodie | Head of Technology Experience (CTO)
>>>>>>>> *SUMO Heavy Industries*
>>>>>>>> Digital Commerce Consulting
>>>>>>>> e: robert@sumoheavy.com
>>>>>>>> t: @sumoheavy
>>>>>>>> p: (917) 525-2606
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Robert Brodie | Head of Technology Experience (CTO)
>>>>>>> *SUMO Heavy Industries*
>>>>>>> Digital Commerce Consulting
>>>>>>> e: robert@sumoheavy.com
>>>>>>> t: @sumoheavy
>>>>>>> p: (917) 525-2606
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Robert Brodie | Head of Technology Experience (CTO)
>>>>>> *SUMO Heavy Industries*
>>>>>> Digital Commerce Consulting
>>>>>> e: robert@sumoheavy.com
>>>>>> t: @sumoheavy
>>>>>> p: (917) 525-2606
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Robert Brodie | Head of Technology Experience (CTO)
>>>>> *SUMO Heavy Industries*
>>>>> Digital Commerce Consulting
>>>>> e: robert@sumoheavy.com
>>>>> t: @sumoheavy
>>>>> p: (917) 525-2606
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Robert Brodie | Head of Technology Experience (CTO)
>>>> *SUMO Heavy Industries*
>>>> Digital Commerce Consulting
>>>> e: robert@sumoheavy.com
>>>> t: @sumoheavy
>>>> p: (917) 525-2606
>>>>
>>>>
>>>
>>>
>>> --
>>> Robert Brodie | Head of Technology Experience (CTO)
>>> *SUMO Heavy Industries*
>>> Digital Commerce Consulting
>>> e: robert@sumoheavy.com
>>> t: @sumoheavy
>>> p: (917) 525-2606
>>>
>>
>>
>
>
> --
> Robert Brodie | Head of Technology Experience (CTO)
> *SUMO Heavy Industries*
> Digital Commerce Consulting
> e: robert@sumoheavy.com
> t: @sumoheavy
> p: (917) 525-2606
>



--
Robert Brodie | Head of Technology Experience (CTO)
*SUMO Heavy Industries*
Digital Commerce Consulting
e: robert@sumoheavy.com
t: @sumoheavy
p: (917) 525-2606