Merge pull request 'Fix bug 501 - 特殊字符&加入生词库后删除按钮失效' (#81) from Bug501-Hui into master

Reviewed-on: http://121.4.94.30:3000/mrlan/EnglishPal/pulls/81

README.md

@@ -11,15 +11,14 @@ Hui Lan <hui.lan@cantab.net>
 
 
 EnglishPal allows the user to build his list of new English words
-picked from articles selected for him according his vocabulary level.
+picked from articles selected for him to read according his vocabulary level.  EnglishPal will determine a user's vocabulary level based on his picked words.  After that, it will recommend articles for him to read, in order to booster his English vocabulary furthermore.
 
 
-## Run it on a local machine
-
+## Run on your own laptop
 
 `python3 main.py`
 
-Make sure you have the SQLite database file in `app/static` (see below).
+Make sure you have put the SQLite database file in the path `app/static` (see below).
 
 
 ## Run it as a Docker container
@@ -29,32 +28,32 @@ Assuming that docker has been installed and that you are a sudo user (i.e., sudo
 
 `sudo ./build.sh`
 
-Open your favourite Internet browser and enter this URL address: `http://ip-address:90`.
+Open your favourite Internet browser and enter this URL address: `http://ip-address:90`.  Note: you must update the variable `DEPLOYMENT_DIR` in `build.sh`.
 
 ### Explanation on the commands in build.sh
 
-My steps for deploying English on the server.
+My steps for deploying English on a Ubuntu server.
 
 - ssh to ubuntu@118.*.*.118
 
-- cd to /home/lanhui/englishpal2/EnglishPal
+- cd to `/home/lanhui/englishpal2/EnglishPal`
 
 - Stop all docker service: `sudo service docker restart`.  If you know the docker container ID, then the above command is an overkill.  Use the following command instead: `sudo docker stop ContainerID`.  You could get all container IDs with the following command: `sudo docker ps`
 
-- Rebuild container. Run the following command to rebuild a docker image after the code gets updated: `sudo docker build -t englishpal .`
+- Rebuild container. Run the following command to rebuild a docker image each time after the source code gets updated: `sudo docker build -t englishpal .`
 
-- Run the application: `sudo docker run -d -p 90:80 -v /home/lanhui/englishpal2/EnglishPal/app/static/frequency:/app/static/frequency -t englishpal`. If you use `sudo docker run -d -p 90:80 -t englishpal`, data will be lost after terminating the program.
+- Run the application: `sudo docker run -d -p 90:80 -v /home/lanhui/englishpal2/EnglishPal/app/static/frequency:/app/static/frequency -t englishpal`. If you use `sudo docker run -d -p 90:80 -t englishpal`, data will be lost after terminating the program.  If you want to automatically restart the docker image after each system reboot, add the option `--restart=always` after `docker run`.
 
-- Save space: `sudo docker system prune -a -f`
+- Save disk space: `sudo docker system prune -a -f`
+
+`build.sh` contains all the above commands.  Run "sudo ./build.sh" to rebuild and start the web application.
 
 
-### Other useful docker commands
+#### Other useful docker commands
 
 - `sudo docker ps -a`
 
-- `sudo docker logs image_name`, where image_name could be obtained from `sudo docker ps`.
-
-`build.sh` contains all the above commands.  Run "sudo ./build.sh" to rebuild and run the web application.
+- `sudo docker logs image_name`, where `image_name` could be obtained from `sudo docker ps`.
 
 
 
@@ -68,6 +67,10 @@ All articles are stored in the `article` table in a SQLite file called
 
 To add articles, open and edit `app/static/wordfreqapp.db` using DB Browser for SQLite (https://sqlitebrowser.org).
 
+### Extending an account's expiry date
+
+By default, an account's expiry is 30 days after first sign-up.  To extend account's expiry date, open and edit `user` table in `app/static/wordfreqapp.db`.  Simply update field `expiry_date`.
+
 ### Exporting the database
 
 Export wordfreqapp.db to wordfreqapp.sql using the following commands:
@@ -92,33 +95,31 @@ sqlite3 wordfreqapp.db`.  Delete wordfreqapp.db first if it exists.
 ### Uploading wordfreqapp.db to the server
 
 
-`pscp wordfreqapp.db lanhui@118.*.*.118:/home/lanhui/englishpal/app/static`
+`pscp wordfreqapp.db lanhui@118.*.*.118:/home/lanhui/englishpal2/EnglishPal/app/static`
 
 
 
 ## Feedback
 
-We welcome feedback on EnglishPal.
+We welcome feedback on EnglishPal.  Feedback examples:
 
-### Respondent 1
+### Feedback 1
+
+- "Need a phone app.  I use phone a lot.  You cannot ask students to use computers."
 
 
-"Need a phone app.  I use phone a lot.  You cannot ask students to use computers."
-
-Can take a picture for text.  Automatic translation.
-
-### Respondent 2
+### Feedback 2
 
 
-“成为会员”改成“注册”
+- “成为会员”改成“注册”
 
-“登出”改成“退出”
+- “登出”改成“退出”
 
-“收集生词吧”改成“生词收集栏”
+- “收集生词吧”改成“生词收集栏”
 
-“不要自动显示下一篇”
+- 不要自动显示下一篇
 
-需要有“上一篇”、“下一篇”
+- 需要有“上一篇”、“下一篇”按钮。
 
 
 
@@ -137,7 +138,7 @@ EnglishPal's bugs and improvement suggestions are recorded in [Bugzilla](http://
 - Usability testing
 
 
-## Improvements made by contributors
+## Improvements made by contributors (incomplete list)
 
 
 ### 朱文绮
@@ -159,7 +160,6 @@ too many words that they already know, on the other hand, it can
 reduce unnecessary memory occupied by the database, in addition, it
 can also improve the simplicity of the page.
 
-More information at: http://118.25.96.118/kanboard/?controller=TaskViewController&action=readonly&task_id=736&token=81a561da57ff7a172da17a480f0d421ff3bc69efbd29437daef90b1b8959
 
 
 ### 占健豪
@@ -181,4 +181,14 @@ Demo video link: https://b23.tv/QuB77m
 Bug report: http://118.25.96.118/bugzilla/show_bug.cgi?id=215
 
 
-*Last modified on 2021-10-17*
+
+### 丁锐
+
+修复了以下漏洞
+
+漏洞：新用户在创建账号时，不需要输入确定密码也可以注册成功，并且新账户可以正常使用。
+
+Bug report: http://118.25.96.118/bugzilla/show_bug.cgi?id=489
+
+
+*Last modified on 2023-01-30*

app/Login.py

@@ -1,6 +1,6 @@
 import hashlib
 import string
-from datetime import datetime
+from datetime import datetime, timedelta
 from UseSqlite import InsertQuery, RecordQuery
 
 path_prefix = '/var/www/wordfreq/wordfreq/'
@@ -23,7 +23,7 @@ def verify_user(username, password):
 
 def add_user(username, password):
     start_date = datetime.now().strftime('%Y%m%d')
-    expiry_date = '20221230'
+    expiry_date = (datetime.now() + timedelta(days=30)).strftime('%Y%m%d') # will expire after 30 days
     # 将用户名和密码一起加密，以免暴露不同用户的相同密码
     password = md5(username + password)
     rq = InsertQuery(path_prefix + 'static/wordfreqapp.db')

app/account_service.py

@@ -5,7 +5,6 @@ from Login import check_username_availability, verify_user, add_user, get_expiry
 # 初始化蓝图
 accountService = Blueprint("accountService", __name__)
 
-
 ### Sign-up, login, logout ###
 @accountService.route("/signup", methods=['GET', 'POST'])
 def signup():
@@ -20,6 +19,7 @@ def signup():
         # POST方法需判断是否注册成功，再根据结果返回不同的内容
         username = escape(request.form['username'])
         password = escape(request.form['password'])
+        password2 = escape(request.form['password2'])
         
         #! 添加如下代码为了过滤注册时的非法字符
         warn = WarningMessage(username)
@@ -32,6 +32,8 @@ def signup():
             return render_template('signup.html')
         elif len(password.strip()) < 4: # 密码过短
             return '密码过于简单。'
+        elif password != password2:
+            return '确认密码与输入密码不一致！'
         else: # 添加账户信息
             add_user(username, password)
             verified = verify_user(username, password)
@@ -48,6 +50,7 @@ def signup():
                 return '用户名密码验证失败。'
 
 
+
 @accountService.route("/login", methods=['GET', 'POST'])
 def login():
     '''

app/static/frequency/README.txt

@@ -0,0 +1,5 @@
+This folder holds users' vocabulary files.
+Each file ends with .pickle.
+For example, mrlan.pickle is the vocabulary file for user mrlan.
+
+

app/static/js/fillword.js

@@ -62,3 +62,7 @@ function onReadClick() {
 function onChooseClick() {
     isChoose = !isChoose;
 }
+
+function stopRead() {
+    reader.cancel();
+}

app/static/js/highlight.js

@@ -22,13 +22,21 @@ function getWord() {
 
 function highLight() {
     if (!isHighlight) return;
-    let articleContent = document.getElementById("article").innerText;
+    let articleContent = document.getElementById("article").innerText; //将原来的.innerText改为.innerHtml，使用innerText会把原文章中所包含的<br>标签去除，导致处理后的文章内容失去了原来的格式
     let pickedWords = document.getElementById("selected-words");  // words picked to the text area
     let dictionaryWords = document.getElementById("selected-words2"); // words appearing in the user's new words list
-    let allWords = pickedWords.value + " " + dictionaryWords.value;
-    const list = allWords.split(" ");
+    let allWords = "";  //初始化allWords的值，避免进入判断后编译器认为allWords未初始化的问题
+    if(dictionaryWords != null){//增加一个判断，检查生词本里面是否为空，如果为空，allWords只添加选中的单词
+        allWords = pickedWords.value + " " + dictionaryWords.value;
+    }
+    else{
+        allWords = pickedWords.value + " ";
+    }
+    const list = allWords.split(" ");//将所有的生词放入一个list中，用于后续处理
     for (let i = 0; i < list.length; ++i) {
         list[i] = list[i].replace(/(^\s*)|(\s*$)/g, ""); //消除单词两边的空字符
+        list[i] = list[i].replace('|', "");
+        list[i] = list[i].replace('?', "");
         if (list[i] !== "" && "<mark>".indexOf(list[i]) === -1 && "</mark>".indexOf(list[i]) === -1) {
 	    //将文章中所有出现该单词word的地方改为：" <mark>" + word + "<mark> "。 正则表达式RegExp()中，"\\s"代表单词前后必须要有空格，以防止只对单词中的部分字符高亮的情况出现。
             articleContent = articleContent.replace(new RegExp("\\s"+list[i]+"\\s", "g"), " <mark>" + list[i] + "</mark> ");
@@ -38,15 +46,15 @@ function highLight() {
 }
 
 function cancelHighlighting() {
-    let articleContent = document.getElementById("article").innerText;
+    let articleContent = document.getElementById("article").innerText;//将原来的.innerText改为.innerHtml，原因同上
     let pickedWords = document.getElementById("selected-words");
     const dictionaryWords = document.getElementById("selected-words2");    
     const list = pickedWords.value.split(" ");    
     if (pickedWords != null) {
         for (let i = 0; i < list.length; ++i) {
             list[i] = list[i].replace(/(^\s*)|(\s*$)/g, "");
-            if (list[i] !== "") {
-                articleContent = articleContent.replace("<mark>" + list[i] + "</mark>", "list[i]");
+            if (list[i] !== "") { //原来判断的代码中，替换的内容为“list[i]”这个字符串，这明显是错误的，我们需要替换的是list[i]里的内容
+                articleContent = articleContent.replace(new RegExp("<mark>"+list[i]+"</mark>", "g"), list[i]);
             }
         }
     }
@@ -55,8 +63,8 @@ function cancelHighlighting() {
         for (let i = 0; i < list2.length; ++i) {
             list2 = dictionaryWords.value.split(" ");
             list2[i] = list2[i].replace(/(^\s*)|(\s*$)/g, "");
-            if (list2[i] !== "") {
-                articleContent = articleContent.replace("<mark>" + list[i] + "</mark>", "list[i]");
+            if (list2[i] !== "") { //原来代码中，替换的内容为“list[i]”这个字符串，这明显是错误的，我们需要替换的是list[i]里的内容
+                articleContent = articleContent.replace(new RegExp("<mark>"+list2[i]+"</mark>", "g"), list2[i]);
             }
         }
     }

app/static/js/word_operation.js

@@ -7,10 +7,20 @@ function familiar(theWord) {
         url:"/" + username + "/" + word + "/familiar",
         success:function(response){
             let new_freq = freq - 1;
-            if(new_freq <1) {
-                $("#p_" + theWord).remove();
+            const allow_move = document.getElementById("move_dynamiclly").checked;
+            if (allow_move) {
+
+                if (new_freq <= 0) {
+                    removeWord(theWord);
+                } else {
+                    renderWord({ word: theWord, freq: new_freq });
+                }
             } else {
-                $("#freq_" + theWord).text(new_freq);
+                if(new_freq <1) {
+                    $("#p_" + theWord).remove();
+                } else {
+                    $("#freq_" + theWord).text(new_freq);
+                }
             }
         }
     });
@@ -25,19 +35,139 @@ function unfamiliar(theWord) {
         url:"/" + username + "/" + word + "/unfamiliar",
         success:function(response){
             let new_freq = parseInt(freq) + 1;
-            $("#freq_" + theWord).text(new_freq);
+            const allow_move = document.getElementById("move_dynamiclly").checked;
+            if (allow_move) {
+                renderWord({ word: theWord, freq: new_freq });
+            } else {
+                $("#freq_" + theWord).text(new_freq);
+            }
         }
     });
 }
 
 function delete_word(theWord) {
     let username = $("#username").text();
-    let word = $("#word_" + theWord).text();
+    let word = theWord.replace('&amp;', '&');
     $.ajax({
         type:"GET",
         url:"/" + username + "/" + word + "/del",
         success:function(response){
-            $("#p_" + theWord).remove();
+            const allow_move = document.getElementById("move_dynamiclly").checked;
+            if (allow_move) {
+                removeWord(theWord);
+            } else {
+                $("#p_" + theWord).remove();
+            }    
         }
     });
 }
+
+/* 
+ * interface Word {
+ *   word: string,
+ *   freq: number
+ * }
+* */
+
+/**
+ * 传入一个词频HTML元素，将其解析为Word类型的对象
+ */
+function parseWord(element) {
+    const word = element
+        .querySelector("a.btn.btn-light[role=button]")  // 获取当前词频元素的词汇元素
+        .innerText  // 获取词汇值;
+    const freq = Number.parseInt(element.querySelector(`#freq_${word}`).innerText);   // 获取词汇的数量
+    return {
+        word,
+        freq
+    };
+}
+
+/**
+ * 使用模板将传入的单词转换为相应的HTML字符串
+*/
+function wordTemplate(word) {
+    // 这个模板应当与 templates/userpage_get.html 中的 <p id='p_${word.word}' class="new-word" > ... </p> 保持一致
+    return `<p id='p_${word.word}' class="new-word" >
+        <a id="word_${word.word}"  class="btn btn-light" href='http://youdao.com/w/eng/${word.word}/#keyfrom=dict2.index'
+           role="button">${word.word}</a>
+        ( <a id="freq_${word.word}" title="${word.word}">${word.freq}</a> )
+        <a class="btn btn-success" onclick="familiar('${word.word}')" role="button">熟悉</a>
+        <a class="btn btn-warning" onclick="unfamiliar('${word.word}')" role="button">不熟悉</a>
+        <a class="btn btn-danger" onclick="delete_word('${word.word}')" role="button">删除</a>
+    </p>`;
+}
+
+/**
+ * 删除某一词频元素
+ * 此处word为词频元素对应的单词
+ */
+function removeWord(word) {
+    // 根据词频信息删除元素
+    word = word.replace('&amp;', '&');
+    const element_to_remove = document.getElementById(`p_${word}`);
+    if (element_to_remove != null) {
+        element_to_remove.remove();
+    }
+}
+
+function renderWord(word) {
+    const container = document.querySelector(".word-container");
+    // 删除原有元素
+    removeWord(word.word);
+    // 插入新元素
+    let inserted = false;
+    const new_element = elementFromString(wordTemplate(word));
+    for (const current of container.children) {
+        const cur_word = parseWord(current);
+        // 找到第一个词频比它小的元素，插入到这个元素前面
+        if (compareWord(cur_word, word) == -1) {
+            container.insertBefore(new_element, current);
+            inserted = true;
+            break;
+        }
+    }
+    // 当word就是词频最小的词时，把他补回去
+    if (!inserted) {
+        container.appendChild(new_element);
+    }
+    // 让发生变化的元素抖动
+    new_element.classList.add("shaking");
+    // 移动到该元素
+    new_element.scrollIntoView({behavior: "smooth", block: "center", inline: "nearest"});
+    // 抖动完毕后删除抖动类
+    setTimeout(() => {
+        new_element.classList.remove("shaking");
+    }, 1600);
+}
+
+/**
+ * 从string中创建一个HTML元素并返回
+ */
+function elementFromString(string) {
+    const d = document.createElement('div');
+    d.innerHTML = string;
+    return d.children.item(0);
+}
+
+/**
+ * 对比两个单词：
+ *  当first小于second时返回-1
+ *  当first等于second时返回0
+ *  当first大于second时返回1
+ */
+function compareWord(first, second) {
+    if (first.freq < second.freq) {
+        return -1;
+    }
+    if (first.freq > second.freq) {
+        return 1;
+    }
+    if (first.word < second.word) {
+        return -1;
+    }
+    if (first.word > second.word) {
+        return 1;
+    }
+    return 0;
+}

app/templates/expiry.html

@@ -5,7 +5,7 @@
     <title>账号过期</title>
 </head>
 <body>
-    <p>您的账号{{ username }}过期。</p>
+    <p>您的账号过期（过期日 {{expiry_date}}）。</p>
     <p>为了提高服务质量，English Pal 收取会员费用， 每天1元。</p>
     <p>请决定你要试用的时间长度，扫描下面支付宝二维码支付。 支付时请注明<i>English Pal Membership Fee</i>。 我们会于12小时内激活账号。</p>
     <p><img src="static/donate-the-author-hidden.jpg" width="120px" alt="支付宝二维码" /></p>

app/templates/signup.html

@@ -17,8 +17,8 @@ You're logged in already! <a href="/logout">Logout</a>.
 
   <form action="/signup" method="POST">
     <p><input type="username" name="username" placeholder="输入用户名" required="required" class="username"></p>
-    <p><input type="password" name="password" placeholder="输入密码" class="password"></p>
-    <p><input type="password" name="password2" placeholder="确认密码" class="password"></p>
+    <p><input type="password" name="password" placeholder="输入密码" required="required" class="password"></p>
+    <p><input type="password" name="password2" placeholder="确认密码" required="required" class="password" ></p>
     <button type="submit" class="btn">注册</button>
   </form>
 

app/templates/userpage_get.html

@@ -19,6 +19,20 @@
     {% endif %}
 
     <title>EnglishPal Study Room for {{ username }}</title>
+
+    <style>
+        .shaking {
+            animation: shakes 1600ms ease-in-out;
+        }
+
+        @keyframes shakes {
+            10%, 90% { transform: translate3d(-1px, 0, 0); }
+            20%, 50% { transform: translate3d(+2px, 0, 0); }
+            30%, 70% { transform: translate3d(-4px, 0, 0); }
+            40%, 60% { transform: translate3d(+4px, 0, 0); }
+            50% { transform: translate3d(-4px, 0, 0); }
+        }
+    </style>
 </head>
 <body>
 <div class="container-fluid">
@@ -67,22 +81,30 @@
     {% endif %}
 
     {% if d_len > 0 %}
-        <p><b>我的生词簿</b></p>
-        {% for x in lst3 %}
-            {% set word = x[0] %}
-            {% set freq = x[1] %}
-            {% if session.get('thisWord') == x[0] and session.get('time') == 1 %}
-                <a name="aaa"></a>
-            {% endif %}
-            <p id='p_{{ word }}' class="new-word" >
-                <a id="word_{{ word }}"  class="btn btn-light" href='http://youdao.com/w/eng/{{ word }}/#keyfrom=dict2.index'
-                   role="button">{{ word }}</a>
-                ( <a id="freq_{{ word }}" title="{{ word }}">{{ freq }}</a> )
-                <a class="btn btn-success" onclick="familiar('{{ word }}')" role="button">熟悉</a>
-                <a class="btn btn-warning" onclick="unfamiliar('{{ word }}')" role="button">不熟悉</a>
-                <a class="btn btn-danger" onclick="delete_word('{{ word }}')" role="button">删除</a>
-            </p>
-        {% endfor %}
+        <p>
+            <b>我的生词簿</b> 
+            <label for="move_dynamiclly">
+                <input type="checkbox" name="move_dynamiclly" id="move_dynamiclly" checked>
+                允许动态调整顺序
+            </label>
+        </p>
+        <a name="aaa"></a>
+        <div class="word-container">
+            {% for x in lst3 %}
+                {% set word = x[0] %}
+                {% set freq = x[1] %}
+                {% if session.get('thisWord') == x[0] and session.get('time') == 1 %}
+                {% endif %}
+                <p id='p_{{ word }}' class="new-word" >
+                    <a id="word_{{ word }}"  class="btn btn-light" href='http://youdao.com/w/eng/{{ word }}/#keyfrom=dict2.index'
+                    role="button">{{ word }}</a>
+                    ( <a id="freq_{{ word }}" title="{{ word }}">{{ freq }}</a> )
+                    <a class="btn btn-success" onclick="familiar('{{ word }}')" role="button">熟悉</a>
+                    <a class="btn btn-warning" onclick="unfamiliar('{{ word }}')" role="button">不熟悉</a>
+                    <a class="btn btn-danger" onclick="delete_word('{{ word }}')" role="button">删除</a>
+                </p>
+            {% endfor %}
+        </div>
         <input id="selected-words2" type="hidden" value="{{ words }}">
     {% endif %}
 </div>

app/templates/userpage_post.html

@@ -20,7 +20,7 @@
     <title>EnglishPal Study Room for {{username}}</title>
 </head>
 <body>
-     <p>勾选不认识的单词</p>
+     <p>取消勾选认识的单词</p>
       <form method="post" action="/{{username}}/mark">
        <input type="submit" name="add-btn" value="加入我的生词簿"/>
        {% for x in lst %}
@@ -30,7 +30,7 @@
             :
             <a href='http://youdao.com/w/eng/{{word}}/#keyfrom=dict2.index' title={{word}}>{{word}}</a>
             ({{x[1]}})
-            <input type="checkbox" name="marked" value="{{word}}">
+            <input type="checkbox" name="marked" value="{{word}}" checked>
         </p>
 
        {% endfor %}

app/user_service.py

@@ -107,7 +107,7 @@ def userpage(username):
     # 用户过期
     user_expiry_date = session.get('expiry_date')
     if datetime.now().strftime('%Y%m%d') > user_expiry_date:
-        return render_template('expiry.html')
+        return render_template('expiry.html', expiry_date=user_expiry_date)
 
     # 获取session里的用户名
     username = session.get('username')

app/wordfreqCMD.py

@@ -39,7 +39,7 @@ def file2str(fname):#文件转字符
 
 
 def remove_punctuation(s): # 这里是s是形参 (parameter)。函数被调用时才给s赋值。
-    special_characters = '\_©~<=>+-/[]*&$%^@.,?!:;#()"“”—‘’{}|《》【】、！￥（）；：？。，' # 把里面的字符都去掉
+    special_characters = '\_©~<=>+-/[]*&$%^@.,?!:;#()"“”—‘’{}|' # 把里面的字符都去掉
     for c in special_characters:
         s = s.replace(c, ' ') # 防止出现把 apple,apple 移掉逗号后变成 appleapple 情况
     s = s.replace('--', ' ')
@@ -103,7 +103,6 @@ if __name__ == '__main__':
     for x in sort_in_descending_order(L):
         print('%s\t%d\t%s' % (x[0], x[1], youdao_link(x[0])))#函数导出
 
-
     # 把频率的结果放result.html中
     make_html_page(sort_in_descending_order(L), 'result.html') 
 
@@ -118,7 +117,6 @@ if __name__ == '__main__':
     # 合并频率
     lst_history = pickle_idea.dict2lst(d)
     d = pickle_idea.merge_frequency(L, lst_history)
-
     pickle_idea.save_frequency_to_pickle(d, 'frequency.p')