Add a feature to remove deleted files entries from TM
ClosedPublic

Authored by sdepiets on Jul 12 2018, 9:08 AM.

Details

Summary

Entries from deleted files are always shown in the Translation Memory. Through an option in the Configuration of the Translation Memory, this feature allows three things :

  • Automatically removing the file from the TM when a missing file's entry is opened through the TM tab
  • Automatically removing the file from the TM when a missing file's entry is opened through the TM suggestions
  • Automatically remove deleted/moved files when rescanning the TM
Test Plan
  • Delete or move a file within a project and rescan the project
  • Delete or move a file within a project and double click one of its entries in the TM tab
  • Delete or move a file within a project and double click one of its entries in the TM suggestions

Diff Detail

Repository
R456 Lokalize
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.
sdepiets requested review of this revision.Jul 12 2018, 9:08 AM
sdepiets created this revision.

Hello,

This is my first feature contribution for a KDE, I apologize in advance if some things are done incorrectly. Any advice is welcome.

mlaurent requested changes to this revision.Jul 12 2018, 9:46 AM
mlaurent added a subscriber: mlaurent.
mlaurent added inline comments.
src/tm/jobs.h
207

space after () and before {

222

it seems a good idea to add a parent to job as RemoveFileJob(...., QObject *parent = nullptr)

224

same

src/tm/tmscanapi.cpp
105 ↗(On Diff #37627)

not necessary to add an empty line in this commit

src/tm/tmtab.cpp
56

it must be in #ifndef NOKDE no ?
same for klocalizedstring.h

This revision now requires changes to proceed.Jul 12 2018, 9:46 AM
sdepiets updated this revision to Diff 37633.Jul 12 2018, 10:14 AM

Fix code formatting

sdepiets updated this revision to Diff 37673.Jul 13 2018, 3:10 AM
sdepiets marked 3 inline comments as done.

Improve performance of file deletion

sdepiets marked an inline comment as done.Jul 13 2018, 3:18 AM

I rewrote the file deletion algorithm because removing each entry from the words table takes too long due to the text research in BLOBs (it took one hour to run on the kde applications .po directory).

Now instead of calling the entry removal, it just deletes the file data in one query per table (source_strings, target_strings, main, files). The entry stays in the words but this shouldn't have any side effect.

For the comment on

#include <klocalizedstring.h>
#include <kmessagebox.h>

being left out of the #ifndef KDE
I followed what was done in editortab.cpp, I'm not sure of the effect of leaving it in or out of the directive but it looked like the safest approach.

src/tm/tmtab.cpp
56

I actually followed what was done in editortab.cpp for this one

sdepiets planned changes to this revision.Jul 13 2018, 3:19 AM
sdepiets requested review of this revision.
sdepiets edited the test plan for this revision. (Show Details)
sdepiets updated this revision to Diff 37674.Jul 13 2018, 3:24 AM

Remove unused parameter

sdepiets updated this revision to Diff 37675.Jul 13 2018, 3:57 AM

Add possibility to manually delete missing files

sdepiets updated this revision to Diff 37676.Jul 13 2018, 4:06 AM

Change test to be consistent with function prototype

sdepiets updated this revision to Diff 37679.Jul 13 2018, 6:41 AM

Remove the deletion choice if file exists and is the current one

mlaurent requested changes to this revision.Jul 19 2018, 5:55 AM
mlaurent added inline comments.
src/tm/tmtab.cpp
56

so it's a bug in source, but even if it's a bug in other file it's not a good idea to continue to make it :)
if we use KDE support we have kmessagebox.h otherwise it will not work.

> for me we need to move in "#ifndef NOKDE"

This revision now requires changes to proceed.Jul 19 2018, 5:55 AM
sdepiets updated this revision to Diff 38058.Jul 19 2018, 6:18 AM

Moved KDE include within the correct directives

sdepiets updated this revision to Diff 38059.Jul 19 2018, 6:21 AM

Delete empty lines

What is a "Deleted file"? A file that did exist and does not exist anymore?

What is the rationale for removing a translation from the translation memory? The fact that a file existed and now doesn't exist anymore doesn't make its translations any less valid/useful as part of the translation memory.

sdepiets marked 3 inline comments as done.Jul 21 2018, 11:11 PM

What is a "Deleted file"? A file that did exist and does not exist anymore?

What is the rationale for removing a translation from the translation memory? The fact that a file existed and now doesn't exist anymore doesn't make its translations any less valid/useful as part of the translation memory.

For instance in the context of the translation of KDE, files sometimes move from one folder to another, or their name change (e.g. all the .desktop.po files changed to ._desktop recently).
In that case all the entries will be duplicated and if you click on an entry in the TM to get more context it will more often than not try to open a non-existing file.

So I think this is rather situationnal and depends on translators preferences, which is why it is disabled by default.
If you think this would be better as a setting at the TM level instead of the app level I probably could move it.

aacid added a comment.Jul 31 2018, 9:47 PM

Do i understand it correctly that we only remove the file on

openFile and on right click?

You mention rescan but i don't see where it happens for rescan.

Also to me it seems that you copied and pasted a sizable chunk of code in three places, can we get that in one place only?

src/prefs/lokalize.kcfg
120

This is not used?

src/tm/tmtab.cpp
535

Nitpicking seems the spacing is a bit off here.

sdepiets updated this revision to Diff 38884.Aug 1 2018, 5:11 AM
sdepiets marked an inline comment as done.

Code simplification, removed useless feature

sdepiets updated this revision to Diff 38885.Aug 1 2018, 5:14 AM

Better spacing

sdepiets marked an inline comment as done.Aug 1 2018, 5:15 AM
aacid added a comment.Aug 16 2018, 8:43 AM

TMTab::openFile() and TMView::deleteFile() still share a sizeable code chunk.

Do you think merging them somehow makes sense?

sdepiets updated this revision to Diff 39845.Aug 16 2018, 9:41 AM

Rebase onto master

sdepiets updated this revision to Diff 39846.Aug 16 2018, 10:10 AM

Code simplification

TMTab::openFile() and TMView::deleteFile() still share a sizeable code chunk.

Do you think merging them somehow makes sense?

I've simplified a bit by moving the file existence check into Project()
It will be complicated to merge more (popup test is different, one of the function has a connect call on the job).

This revision was not accepted when it landed; it landed in state Needs Review.Aug 16 2018, 11:44 AM
This revision was automatically updated to reflect the committed changes.