nodejs.org: falling back to english when page hasn't been translated by phillipj · Pull Request #316 · nodejs/build

phillipj · 2016-01-29T21:53:37Z

Currently translation working groups will have to translate all pages to their respective
language, if not most pages would end in displaying a (english) 404 page. That's a massive
requirement on translation groups, and keeping up with future changes are probably near impossible.

Lowering the barrier and letting translation groups start on the frontpage and work their way
down a couple of page depths, should be alot simpler and more rewarding.

NB! This only affects local development as nodejs.org runs nginx serving strictly static content.
Getting the same behaviour in production requires changes to the nginx config.

Example of the current 404 issue; navigate any of the headers links in the current Korean translation: https://nodejs.org/ko/. I haven't done my dues when it comes to translating (yet), this is one of the main reasons I've decided not to.

jbergstroem · 2016-02-02T00:28:17Z

This will make all 404's redirect to /en/ for the nodejs.org scope (besides dist) - right? Not sure that is the desired effect.

phillipj · 2016-02-02T06:40:39Z

Good question. Got to admit I didn't think about URLs other than the actual
website. I'll dig more into that as soon as I'm back from one week vacation.

On Tuesday, 2 February 2016, Johan Bergström notifications@github.com
wrote:

This will make all 404's redirect to /en/ for the nodejs.org scope
(besides dist) - right? Not sure that is the desired effect.
—
Reply to this email directly or view it on GitHub
#316 (comment).

phillipj · 2016-02-13T21:08:48Z

This will make all 404's redirect to /en/ for the nodejs.org scope (besides dist) - right?

I assume you would rather see it check the /en/ equivalent exist before doing a redirect, correct? Haven't been able to get that working... Got a trick up your sleeve @jbergstroem?

phillipj · 2016-03-02T20:45:47Z

To elaborate on what I've tried to accomplish without luck so far:

location @english_fallback {
set $english_filename '';
if ($request_filename !~* /en/) {
set $english_filename ...; # need string replace??
}
if (-e $english_filename) {
rewrite ^(/\w+/)(.*)$ http://nodejs.org/en/$2;
}
return 404;
}

The biggest challenge here is doing a string replace on the current $request_filename to get the english equivalent need to do the existence check. AFAIK the kind of manual string replace functionality needed, would require nginx_substitutions_filter to be installed.

Anyway the use if is strongly discouraged by the nginx team and community...

jbergstroem · 2016-03-02T21:44:13Z

How about going at it the other way? redirecting all other 'known' languages that doesn't exist to en?

phillipj · 2016-03-07T19:24:34Z

@jbergstroem something like this?

location @english_fallback {
rewrite ^/(it|ko)/(.*)$ http://nodejs.org/en/$2;
return 404;
}

That surely removes the complexity and no if's in sight 👍

phillipj · 2016-03-09T22:02:50Z

@jbergstroem just updated the PR with the simplified version posted above ^^ with explicit known languages as you suggested. Also added the english fallback to the https server directive.

phillipj · 2016-03-23T17:11:41Z

@jbergstroem any thoughts on this after it got updated?

Fishrock123 · 2016-04-11T16:58:45Z

Didn't we have something like this for iojs.org?

Looks like it does work on iojs.org: https://iojs.org/fa/es6.html

Ideally, however, it would not re-write to /en/ so that any future links can still go to the native language? I think? I'm not sure.

Fishrock123 · 2016-04-11T17:02:23Z

Hmm, looks like we just made redirect static files for each page: https://github.com/nodejs/iojs.org/blob/master/source/static/es6.html

Fishrock123 · 2016-04-11T17:24:04Z

As a note, 404's don't even work correctly at the current time: https://nodejs.org/ko/404/ - yet we clearly have one for korean: https://github.com/nodejs/nodejs.org/blob/master/locale/ko/404.md

Sign in to view

 }

 location / {
+ try_files $uri $uri/ @english_fallback;


Fishrock123 · 2016-04-24T18:31:02Z

Ok I spent some time trying to get the localized 404 pages working today.

What I've come up with:

location @english_fallback {if($uri~* ^/(it|ko)/){set$lang$1;}rewrite ^/(it|ko)/(.*)$ /en/$2; # I've used a relative url here for testing, not sure if it should be absolute or not
 # No `return 404;` here, I think that makes our page actually return that status code ontop of serving..}

Where we currently declare error_page 404 /en/404.html;:

error_page @404;# use localized 404 pages if possiblelocation @404{try_files /$lang/404.html /en/404.html;}

Fishrock123 · 2016-04-24T18:31:42Z

@phillipj Hopefully you have time to look at this, I'd definitely like to get it live!

jbergstroem · 2016-04-24T20:35:07Z

I will follow up here shortly too.

phillipj · 2016-04-25T14:44:45Z

@Fishrock123 awesome, thanks for pitching in here! Just pushed an update which contains your localized 404 hackery 👍

Sign in to view

+ if ($uri ~* ^/(it|ko)/) {
+ set $lang $1;
+ }
+ rewrite ^/(it|ko)/(.*)$ /en/$2;


Fishrock123 · 2016-05-02T16:31:06Z

@nodejs/build could some people look at this please? This is really important for website translation and I'd like to get it live ASAP.

Refs nodejs/nodejs.org#490

phillipj · 2016-05-02T18:19:50Z

Rebased to fix merge conflict.

Also decided to not go for relative rewrites to do things as before, and not raising more questions than necessary.

Starefossen · 2016-05-02T18:42:32Z

Looks good to me 👍

jbergstroem · 2016-05-02T22:51:46Z

Sorry for the delay -- I just need to do some performance testing (if's are evil yada yada)

Fishrock123 · 2016-05-03T18:59:59Z

ifs might be evil but I don't know another way to do it. :(

Fishrock123 · 2016-05-13T20:19:30Z

@jbergstroem Can we merge this and if it really turns out to be a problem, undo it and have a single, multi-language 404 page? This is badly blocking translation groups as I understand... I know you're busy. :(

(I don't really know how to benchmark this... if you just want me to run ab or wrk against it I can do that though!)

jbergstroem · 2016-05-31T10:10:19Z

@phillipj I'd suggest having a ubuntu14 machine available and override hosts iojs-www and node-www when you run the playbook then. You'll have to add those hosts to your ssh config.

rvagg · 2016-05-31T14:42:33Z

I've been messing with setting up a test machine for you to play with this on, knowing that our ansible scripts are not entirely solid for the web server this was a good chance to try and address that.

@phillipj I've put your ssh keys from GitHub into root@107.170.115.122 and it's set up almost identically to the nodejs.org server and you can do what you like with it. Stick 107.170.115.122 nodejs.org into your /etc/hosts to make it work like the real thing for you. /home/www/nodejs/ is where the web files are. Run /home/nodejs/build-site.sh nodejs if you want to rebuild the web site from the nodejs.org/node repo. /etc/nginx/sites-available/nodejs.org is where the nginx config for the site is that you can play with. This machine is not entirely devoid of production secrets so don't go liberally sharing access to it, in fact if you want to give someone else access you might want to ask here about that.

For the record: the playbook in this repo for www should be in a better state now, it can be run with ansible-playbook -i <path to inventory file with a [node-www] block and a host listed under it> ansible-playbook.yaml but the tricky bit is that you'll need some production secrets available to do a proper deploy, faking them out as things fail is one way to go, some info about what's needed can be gleaned from resources/secrets/makesecrets.sh too.

rvagg · 2016-05-31T14:43:38Z

@Fishrock123 I put your keys in there too if you're interested in shaving this yak

Fishrock123 · 2016-05-31T15:27:57Z

Noted, looking at it right now.

Fishrock123 · 2016-05-31T15:33:29Z

Hmm, Indeed it does not appear to be working.

Aside: why do we have 2 server configs for both blog.nodejs.org and nodejs.org? It's quite confusing.

rvagg · 2016-05-31T22:40:48Z

@Fishrock123 legacy, you'll note that all of the blog redirects are contained within the blog.nodejs.org part of the config, which leave the main nodejs.org config with much fewer redirects in its flow.

I think that's what you're referring to. Or perhaps you're seeing :80: vs :443. Again, legacy, the ideal situation is to have a minimal :80 but we've yet to fully make that cutover because we've been allowing http access to /dist/ for legacy reasons. We'll get there soon, I think in the next few months in fact.

rvagg · 2016-05-31T22:42:24Z

Also, you're welcome to propose improvements to our nginx config, for both readability and actual efficiency. Just be aware of the legacy constraints, things need to continue to work as they do now. Mostly I'm pretty afraid to mess with nginx config too much because it's a bit like playing wac-a-mole.

https://github.com/nodejs/build/blob/master/setup/www/resources/config/nodejs.org

phillipj · 2016-06-03T14:04:05Z

@rvagg thanks for adding my keys!

After some trial-n-error, it seems like using relative URLs in the fallback rewrites made a positive impact. I just pushed a commit. Those changes has been made on 107.170.115.122 if some of you want to do a quick check, by adding it into /etc/hosts as mentioned in #316 (comment).

phillipj · 2016-06-04T12:27:30Z

Added zh-cn in addition to it|ko cause of recently added translations (e.g. nodejs/nodejs.org#771 & nodejs/nodejs.org#776)

Sign in to view

+ error_page 404 @localized_404;
+
 location / {
+ try_files $uri $uri/ @english_fallback;


rvagg · 2016-06-08T11:07:12Z

sgtm but I'm way too gun shy with this nginx config to be comfortable signing off on it. Can others who are comfortable hacking nginx config have a look and try it out on the test host too? @jbergstroem?

jbergstroem · 2016-06-08T12:37:07Z

I will follow up shortly to confirm that it works as intended. I wasn't a big fan of how variable scopes were used; but it was indeed nifty.

Fishrock123 · 2016-06-13T17:25:43Z

At some point we're going to have to just try it.

Fishrock123 · 2016-06-24T13:26:09Z

... ping again ...

rvagg · 2016-06-27T10:35:07Z

I'm OK with giving this a try, but only if @jbergstroem is +1 too. We should probably do it during a low traffic period, the weekend I guess, as long as we have people around to revert quickly if need be.

jbergstroem · 2016-06-27T10:57:07Z

Not sure which timezone digitalocean runs by, but least amount of traffic occurs in ~13h from now. I can attempt to give it a test around then.

jbergstroem · 2016-06-28T06:43:58Z

This has been running for a bit on nodejs.org now and seems to be working as intended.

Fishrock123 · 2016-06-30T10:53:43Z

Has been running for about 3 days now? I think this can be merged

jbergstroem · 2016-07-04T20:37:37Z

I'll merge this!

phillipj · 2016-07-28T11:00:42Z

@jbergstroem any progress on merging this? nodejs.org just got a new locale (es / spanish), so it would be nice to get this merged before opening a new PR adding spanish as well.

jbergstroem · 2016-07-28T13:32:14Z

@phillipj I'll add es/spanish.

Fall back to english if a page hasn't been translated. Refs: nodejs/nodejs.org#490 PR-URL: #316 Reviewed-By: Johan Bergström <bugs@bergstroem.nu> Reviewed-By: Rod Vagg <rod@vagg.org>

jbergstroem · 2016-07-28T13:44:53Z

Merged (with the addition of es) in 9488a61.

phillipj · 2016-07-28T16:31:59Z

Awesome 👍

On Thursday, 28 July 2016, Johan Bergström notifications@github.com wrote:

Merged (with the addition of es) in 9488a61
9488a61
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#316 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABLLE96B7JlTVHmgHr8BZD05RrK_q6jQks5qaLJYgaJpZM4HPlRm
.

phillipj mentioned this pull request Feb 1, 2016
Falling back to english when page hasn't been translated locally nodejs/nodejs.org#490
Merged

phillipj force-pushed the fallback-to-english branch 2 times, most recently from 0a34b07 to 224dcdbCompare March 9, 2016 21:59

Fishrock123 reviewed Apr 11, 2016
View reviewed changes

phillipj force-pushed the fallback-to-english branch 2 times, most recently from ebf0e0b to e8d291eCompare April 25, 2016 14:42

phillipj reviewed Apr 25, 2016
View reviewed changes

Comment threadsetup/www/resources/config/nodejs.org Outdated
if ($uri ~* ^/(it|ko)/) {
set $lang $1;
}
rewrite ^/(it|ko)/(.*)$ /en/$2;
This comment was marked as off-topic.
Sign in to view

nodejs.org: falling back to english when page hasn't been translated.
ef8b01b
Refs nodejs/nodejs.org#490

phillipj force-pushed the fallback-to-english branch from e8d291e to ef8b01bCompare May 2, 2016 18:17

Use relative rewrite URLs in fallbacks
c7d58f4

Add locale zh-cn
0704893

rvagg reviewed Jun 8, 2016
View reviewed changes

Fishrock123 mentioned this pull request Jun 28, 2016
Discuss moving to nodejs.org/ko/ now that a language issue was fixed nodejs/nodejs-ko#381
Open

Fishrock123 mentioned this pull request Jul 4, 2016
Translation nodejs/nodejs.org#807
Closed

jbergstroem closed this Jul 28, 2016

phillipj mentioned this pull request Aug 3, 2016
locale: add Ukrainian language nodejs/nodejs.org#844
Merged

Uh oh!

Uh oh!

Conversation

phillipj commented Jan 29, 2016

Uh oh!

jbergstroem commented Feb 2, 2016

Uh oh!

phillipj commented Feb 2, 2016

Uh oh!

phillipj commented Feb 13, 2016

Uh oh!

phillipj commented Mar 2, 2016

Uh oh!

jbergstroem commented Mar 2, 2016

Uh oh!

phillipj commented Mar 7, 2016

Uh oh!

phillipj commented Mar 9, 2016

Uh oh!

phillipj commented Mar 23, 2016

Uh oh!

Fishrock123 commented Apr 11, 2016

Uh oh!

Fishrock123 commented Apr 11, 2016

Uh oh!

Fishrock123 commented Apr 11, 2016

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

Fishrock123 commented Apr 24, 2016

Uh oh!

Fishrock123 commented Apr 24, 2016

Uh oh!

jbergstroem commented Apr 24, 2016

Uh oh!

phillipj commented Apr 25, 2016

Uh oh!

This comment was marked as off-topic.

Uh oh!

Fishrock123 commented May 2, 2016

Uh oh!

phillipj commented May 2, 2016

Uh oh!

Starefossen commented May 2, 2016

Uh oh!

jbergstroem commented May 2, 2016

Uh oh!

Fishrock123 commented May 3, 2016

Uh oh!

Fishrock123 commented May 13, 2016

Uh oh!

jbergstroem commented May 31, 2016

Uh oh!

rvagg commented May 31, 2016

Uh oh!

rvagg commented May 31, 2016

Uh oh!

Fishrock123 commented May 31, 2016

Uh oh!

Fishrock123 commented May 31, 2016

Uh oh!

rvagg commented May 31, 2016

Uh oh!

rvagg commented May 31, 2016

Uh oh!

phillipj commented Jun 3, 2016

Uh oh!

phillipj commented Jun 4, 2016

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.

Uh oh!

This comment was marked as off-topic.