Create Task
Maniphest T5429

math_check_online run on localhost
Closed, ResolvedPublic
Actions

Description

This plugin is marvellous: it allows the importer to check the correctness of the math blocks, sending them to the restbase service of wikitolearn.org .
Unfortunately this requires a huuge number of requests to the restbase service, and this takes time (a couple of equations per second when it is fast). The problem is that as soon as the connection has a slight problem, the connection to the restbase service times out, and texla crashes, without keeping a cache of the checked equations.
One solution may be building a cache of the checked equations, but I understand that this may be hard.
Another solution, proposed by @tomaluca, is changing the server that we use to check the math. We may use a local installation of WTL for checking it.
At the current stage, it seems that the only reference to the server that runs restbase in math_check_online is in ./texla/Renderers/plugins/math_check_online.py and it is

url_check = 'http://restbase.wikitolearn.org/pool.wikitolearn.org/v1/media/math/check/tex'

I tried changing it into

url_check = 'http://restbase.tuttorotto.biz/pool.tuttorotto.biz/v1/media/math/check/tex'

and, with a running local environment of WTL, it works.
The problem is that this requires an intenet connection to resolve tuttorotto.biz, and this does not sove the problem, we are still dependent from the internet connection, which may have troubles.
I tried changing that line into

url_check = 'http://127.0.0.1/127.0.0.1/v1/media/math/check/tex'

But again this does not work offline. Understanding why is a bit harder now, and i have not found the cause yet.
I have pasted the errors that texla spits out here .

The problem may be that there still is some rference to wikitolearn.org somewhere, but I do not know where.

requests.packages.urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='www.wikitolearn.org', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f6f252ebc50>: Failed to establish a new connection: [Errno -2] Name or service not known',))

It would be very nice to solve this problem, since it would:

  • make the math check more stable (not dependant on the stability internet connection)
  • speed the math checking

EDIT: I tried changing the /etc/hosts file adding there the resolution of the www.wikitolearn.org domain adding athe end of the file

127.0.0.1 www.wikitolearn.org wtl

but this does not works, since there are some problems with the certificates, you can see them here

mapellidario updated the task description. (Show Details)Feb 20 2017, 2:21 PM
tomaluca added a comment.Feb 20 2017, 2:31 PM
  1. run wtlhome
  2. apply the patch
diff --git a/texla/Renderers/plugins/math_check_online.py b/texla/Renderers/plugins/math_check_online.py
index d3d18f5..c625fa3 100644
--- a/texla/Renderers/plugins/math_check_online.py
+++ b/texla/Renderers/plugins/math_check_online.py
@@ -5,7 +5,7 @@ from os import path
 
 
 def request_formula(tex):
-    url_check = 'http://restbase.wikitolearn.org/pool.wikitolearn.org/v1/media/math/check/tex'
+    url_check = 'http://restbase.tuttorotto.biz/pool.tuttorotto.biz/v1/media/math/check/tex'
     header = {
         'Accept': 'application/json',
         'Content-Type': 'application/x-www-form-urlencoded'

may be a good idea to have a config for the domain somewhere in texla

mapellidario added a comment.Feb 20 2017, 2:36 PM

Yes, I have done it, but i would like it not to be dependent on the DNS resolution, since it still would make the script dependent on the internet connection, and the problem is not solved

davidev reassigned this task from davidev to tomaluca.Feb 24 2017, 11:22 AM
davidev added a subscriber: davidev.
tomaluca added a comment.Feb 24 2017, 12:17 PM

our system is name based, some level of dns is needed, you could use the /etc/hosts to hack around this, or the system could use raw mathoid to check the rendering, in this way you will only have to run the mathoid docker and point the math check to 127.0.0.1:<someport> skipping all name based stuff.

tomaluca closed this task as Resolved.Feb 24 2017, 4:04 PM
mapellidario added a comment.Feb 24 2017, 11:31 PM

Ok, i found a config that wokrs offline:

/etc/hosts

[...]
# wikitolearn
127.0.0.1       pool.tuttorotto.biz          tuttorotto
127.0.0.1       restbase.tuttorotto.biz      tuttorotto
[...]

texla/Renderers/plugins/math_check_online.py

[...]
def request_formula(tex):
#    url_check = 'http://restbase.wikitolearn.org/pool.wikitolearn.org/v1/media/math/check/tex'
    url_check = 'http://restbase.tuttorotto.biz/pool.tuttorotto.biz/v1/media/math/check/tex'
[...]

It still is not clear to me why if i set url_check = 'http://127.0.0.1/127.0.0.1/v1/media/math/check/tex', it did not work offline, since it tried to reach wikitolearn.org. Still, this is more of a curiosity then a question that requires an urgent answer.