Bug 142404 - FILEOPEN DOCX TAB_OVER_MARGIN not needed for compatibiltymode >= 15 (almost fixed - see comment 24)
Summary: FILEOPEN DOCX TAB_OVER_MARGIN not needed for compatibiltymode >= 15 (almost f...
Status: NEW
Alias: None
Product: LibreOffice
Classification: Unclassified
Component: Writer (show other bugs)
Version:
(earliest affected)
7.2.0.0.alpha0+
Hardware: All All
: medium normal
Assignee: Not Assigned
URL:
Whiteboard: target:7.2.0 target:7.3.0 target:7.2....
Keywords: filter:docx
Depends on:
Blocks: DOCX-Paragraph DOCX-compatibilityMode-15
  Show dependency treegraph
 
Reported: 2021-05-21 07:27 UTC by NISZ LibreOffice Team
Modified: 2024-05-21 17:55 UTC (History)
5 users (show)

See Also:
Crash report or crash signature:


Attachments
attachment 90834 converted to compatibilitymode 15 with Word 2019 (20.55 KB, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
2021-05-21 07:27 UTC, NISZ LibreOffice Team
Details
attachment 90834 converted to compatibilitymode 14 with Word 2010 (25.91 KB, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
2021-05-21 07:28 UTC, NISZ LibreOffice Team
Details
Screenshot of the compatibilitymode 15 document side by side in Word and Writer (120.35 KB, image/png)
2021-05-21 07:29 UTC, NISZ LibreOffice Team
Details
Screenshot of the compatibilitymode 14 document side by side in Word and Writer (137.90 KB, image/png)
2021-05-21 07:29 UTC, NISZ LibreOffice Team
Details
tabStopsTripleSpacingc11.doc: shows general problem with IgnoreTabsAndBlanksForLineCalculation that is not related to compatibility mode. (23.00 KB, application/msword)
2021-05-26 09:54 UTC, Justin L
Details
tabStopsTripleSpacingc11_word2003.pdf: two pages long with triple-spaced tab lines (8.88 KB, application/pdf)
2021-05-26 10:55 UTC, Justin L
Details
tabOverflow.odt: Various tabOverflow scenarios. (14.77 KB, application/vnd.oasis.opendocument.text)
2021-06-01 06:35 UTC, Justin L
Details
tabOverflow_comparison.pdf: Writer, Word 2016 in compatibile mode, and then in native mode (16.47 KB, application/pdf)
2021-06-01 06:47 UTC, Justin L
Details
tabOverflowB.odt: tabOverSpacing still applies in c15 though (14.04 KB, application/vnd.oasis.opendocument.text)
2021-06-01 13:22 UTC, Justin L
Details
tabOverSpacing_odd_c12.docx: MS Word 2016 renders this very strangely. (5.95 KB, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
2021-06-03 07:30 UTC, Justin L
Details
tabOverSpacing_odd_c12.pdf: How Word 2016 renders it. (12.08 KB, application/pdf)
2021-06-03 07:58 UTC, Justin L
Details
A regression testcase for 89e734 (13.69 KB, application/vnd.openxmlformats-officedocument.wordprocessingml.document)
2024-05-21 16:56 UTC, Vasily Melenchuk (CIB)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description NISZ LibreOffice Team 2021-05-21 07:27:22 UTC
Created attachment 172218 [details]
attachment 90834 [details] converted to compatibilitymode 15 with Word 2019

Attached example file was converted from attachment 90834 [details] with Word 2019 to the latest docx format.
Another example file was converted from the same attachment with Word 2010.
These look differently in Word, but not in Writer: In newer Word versions the tabs at the end of the line are not swallowed, but broken into new lines.

Steps to reproduce:
    1. Open attached file

Actual results:
The four tabs at the end of the line are not broken into new lines.

Expected results:
The four tabs at the end of the line should be broken into new lines. 
But not with the 2010-compatibility mode file, there any amount of extra tabs should be swallowed, per: https://bugs.documentfoundation.org/show_bug.cgi?id=107476#c7 

LibreOffice details:
Version: 7.2.0.0.alpha1+ (x64) / LibreOffice Community
Build ID: eda109d5d91e82adb675ebf6db952bf6d82b3391
CPU threads: 4; OS: Windows 10.0 Build 18363; UI render: default; VCL: win
Locale: hu-HU (hu_HU); UI: en-US
Calc: CL
Comment 1 NISZ LibreOffice Team 2021-05-21 07:28:37 UTC
Created attachment 172219 [details]
attachment 90834 [details] converted to compatibilitymode 14 with Word 2010
Comment 2 NISZ LibreOffice Team 2021-05-21 07:29:09 UTC
Created attachment 172220 [details]
Screenshot of the compatibilitymode 15 document side by side in Word and Writer
Comment 3 NISZ LibreOffice Team 2021-05-21 07:29:28 UTC
Created attachment 172221 [details]
Screenshot of the compatibilitymode 14 document side by side in Word and Writer
Comment 4 Justin L 2021-05-21 08:50:34 UTC
I think Miklos will be interested in hearing about this compat change.  CC'd.

I confirmed that just changing word/settings.xml compatibilityMode from 14 to 15 is enough to make this rather dramatic change.
Comment 5 Miklos Vajna 2021-05-21 09:06:42 UTC
Nice, for once Word giving up its own stupid behavior for new docs. :-)
Comment 6 Justin L 2021-05-24 12:46:27 UTC
It looks like this is not just TabOverMargin, but also TabOverflow. Plus something else which we don't handle yet - the first tab should be full page width, so that it jumps to a new line if there are any characters before the tab? [Trading one bad implementation for another?]

It looks like we could set c14 values in  SettingsTable::ApplyProperties and remove TabOverMargin from filter/Writerfilter::setTargetDocument (and set TabOverflow to false here).

tdf90681-1.odt from ooxmlexport5 is one barely-noticable existing unit test that demonstrates the c15 behaviour. (The footer is two empty lines instead of one.)
Comment 7 Justin L 2021-05-25 11:51:20 UTC
Sometimes compat settings are hijacked for another purpose. However, I didn't notice that for either tabOverMargin or tabOverflow. Good.
Comment 8 Justin L 2021-05-26 09:54:01 UTC
Created attachment 172352 [details]
tabStopsTripleSpacingc11.doc: shows general problem with IgnoreTabsAndBlanksForLineCalculation that is not related to compatibility mode.

(In reply to Justin L from comment #6)
> Plus something else which we don't handle yet - the first tab should be full
> page width, so that it always takes up a full line.
It looks like SwTabPortion::PreFormat (either bFull or PrtWidth(rInf.Width()) can handle this.

One thing I hadn't noticed before is that some tab-only-lines are double-spaced and some are single spaced, but in LO they show as single-spaced. That started in LO 4.0 range https://cgit.freedesktop.org/libreoffice/core/log/?id=c26e9fc152f6a52fd95a7ef2a3e6ffbd9439ca4c&qt=range&q=179a6db61ee30cf776747802f06edeef45fec461..5f91f8a368343d8921a01edb7359cd300892f09d).

The specific commit is "n#779627: added quite some compat options from the ww8 filter on writerfilter" https://cgit.freedesktop.org/libreoffice/core/commit/?id=355d25eac764713f4d52eac801ade6e2ff1deab0, because of IgnoreTabsAndBlanksForLineCalculation.
Comment 9 Justin L 2021-05-26 10:55:31 UTC
Created attachment 172354 [details]
tabStopsTripleSpacingc11_word2003.pdf: two pages long with triple-spaced tab lines
Comment 10 Commit Notification 2021-05-28 15:07:04 UTC
Justin Luth committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/89e7341025b607491c90efdb74708e63d875c1e5

tdf#142404 sw compat layout: ignore blank size only on one-liners

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 11 Justin L 2021-06-01 06:35:02 UTC
Created attachment 172516 [details]
tabOverflow.odt: Various tabOverflow scenarios.

When we have TabOverMargin, that also involves TabOverflow. So what does Microsoft do in this case now?

Note that LibreOffice changed its OWN behaviour in 2011, so that tab-fill would overflow the margin, just like MS Word did. But now DOCX has changed, so do we change LO's OWN behaviour back again just for DOCX and therefore be less compatible with DOC/RTF?

See the discussion about this in https://gerrit.libreoffice.org/c/core/+/116336.
Comment 12 Justin L 2021-06-01 06:47:56 UTC
Created attachment 172517 [details]
tabOverflow_comparison.pdf: Writer, Word 2016 in compatibile mode, and then in native mode

When LO 6.4 saves tabOverflow.ODT as a DOCX, Word opens it in compatible mode. When LO 7.0 saves it as a DOCX, Word opens it in native mode. Writer displays it somewhere in the middle...
Comment 13 Commit Notification 2021-06-01 12:58:03 UTC
Justin Luth committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/29bd00f7628e7a54e69cabcc7e2a1792c24aa55c

tdf#142404 DOCX c15: TabOverMargin no longer true in 2013+

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 14 Justin L 2021-06-01 13:22:38 UTC
Created attachment 172553 [details]
tabOverflowB.odt: tabOverSpacing still applies in c15 though

I'm going to delay backporting to 7.1 until I've had a chance to try to figure out how to allow "tab over right spacing" as seen in this example document. That still applies in Word 2016 (but is lost in LO when turning off tabOverMargin).
Comment 15 Justin L 2021-06-03 07:30:53 UTC
Created attachment 172589 [details]
tabOverSpacing_odd_c12.docx: MS Word 2016 renders this very strangely.

I just wanted to preserve this DOCX that makes NO LOGICAL SENSE in how MS Word renders it. This is in compatibility mode.

The super strange line is "A left tab positioned at 15cm" which for whatever reason breaks into multiple lines - the most bizarre being the one starting with "1 and forever". What could possibly have induced that to wrap at that point?

LO also handles it really poorly. The extra defined tab at 2cm seems to cause problems for initial layout.
Comment 16 Justin L 2021-06-03 07:58:24 UTC
Created attachment 172590 [details]
tabOverSpacing_odd_c12.pdf: How Word 2016 renders it.
Comment 17 Justin L 2021-06-03 12:52:16 UTC
Proposed patch that seems to handle the tab over paragraph-margins nicely. http://gerrit.libreoffice.org/c/core/+/116667
Comment 18 Commit Notification 2021-06-08 13:24:34 UTC
Justin Luth committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/a2d8a737f41cbb0ed8a51a99b1c233b25e433d1c

tdf#142404 DOCX c15: add compat flag TabOverSpacing

It will be available in 7.2.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 19 Commit Notification 2021-06-15 14:05:51 UTC
Justin Luth committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/39251e461bfe628da875bb17e91dc374f137eec5

tdf#142404 DOCX c15: allow autoTabStop after TabOverSpacing

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 20 Commit Notification 2021-06-16 14:54:57 UTC
Justin Luth committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/c54de8cca177154d267238ed6d0b6cb4f3f86c8d

tdf#142404 DOCX c15: line full before out-of-bounds TabOverSpacing

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 21 Commit Notification 2021-06-17 04:22:35 UTC
Justin Luth committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/3f5311271ea6e41583c581b8a0172e322f5a179a

tdf#142404 DOCX c15: fix centering when TabOverSpacing

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 22 Commit Notification 2021-06-19 17:57:24 UTC
Justin Luth committed a patch related to this issue.
It has been pushed to "libreoffice-7-2":

https://git.libreoffice.org/core/commit/d7a7ac3581290f9d4820d01d0a64269fc47d00f6

tdf#142404 DOCX c15: fix centering when TabOverSpacing

It will be available in 7.2.0.0.beta2.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 23 Commit Notification 2021-06-19 18:14:55 UTC
Justin Luth committed a patch related to this issue.
It has been pushed to "master":

https://git.libreoffice.org/core/commit/b85491e40ccb83ff78c7c4b2c0d535eafa5d23ed

tdf#142404 DOCX c15: handle remaining TabOverSpacing tabs

It will be available in 7.3.0.

The patch should be included in the daily builds available at
https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
information about daily builds can be found at:
https://wiki.documentfoundation.org/Testing_Daily_Builds

Affected users are encouraged to test the fix and report feedback.
Comment 24 Justin L 2021-06-21 10:50:21 UTC
I'm tempted to mark this as fixed, even though comment 0's 40787_TabsAtEndOfParagraphC15.docx isn't yet identical to Word 2016.

That's because Word does REALLY STRANGE THINGS. If the tab is connected directly to a word, the word is pulled down, and then the tab follows - usually again on another line. When - as in this case - the tab follows some whitespace, then only the tab starts a new line, and it does NOT pull down any of the whitespace.

So there is still a difference, since I don't have the first tabstop always start on a new line. But that doesn't seem logical to do that, and there seem to be some bugs in MS implementation, so we probably shouldn't try TOO hard to exactly match what we see in Word.
Comment 25 QA Administrators 2023-08-21 03:05:36 UTC Comment hidden (obsolete, spam)
Comment 26 Vasily Melenchuk (CIB) 2024-05-21 16:56:57 UTC
Created attachment 194249 [details]
A regression testcase for 89e734

(In reply to Commit Notification from comment #10)
> Justin Luth committed a patch related to this issue.
> It has been pushed to "master":
> 
> https://git.libreoffice.org/core/commit/
> 89e7341025b607491c90efdb74708e63d875c1e5
> 
> tdf#142404 sw compat layout: ignore blank size only on one-liners
> 
> It will be available in 7.2.0.
> 
> The patch should be included in the daily builds available at
> https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
> information about daily builds can be found at:
> https://wiki.documentfoundation.org/Testing_Daily_Builds
> 
> Affected users are encouraged to test the fix and report feedback.

Seems these changes are causing (In reply to Commit Notification from comment #10)
> Justin Luth committed a patch related to this issue.
> It has been pushed to "master":
> 
> https://git.libreoffice.org/core/commit/
> 89e7341025b607491c90efdb74708e63d875c1e5
> 
> tdf#142404 sw compat layout: ignore blank size only on one-liners
> 
> It will be available in 7.2.0.
> 
> The patch should be included in the daily builds available at
> https://dev-builds.libreoffice.org/daily/ in the next 24-48 hours. More
> information about daily builds can be found at:
> https://wiki.documentfoundation.org/Testing_Daily_Builds
> 
> Affected users are encouraged to test the fix and report feedback.

Seems this patch is causing regressions in MS docs layout. See attached file.

Commit message states:
    If the paragraph is longer than one line,
    then do not IgnoreBlanksAndTabsForLineHeightCalculation,
    which was introduced in LO 4.0.

But I can't reproduce this behavior with Word 365. Can you give a scenario to test this?
Comment 27 Justin L 2024-05-21 17:55:57 UTC
(In reply to Vasily Melenchuk (CIB) from comment #26)
> But I can't reproduce this behavior with Word 365. Can you give a scenario
> to test this?
Sure. There is a unit test document attached to the patch you can test with.

My guess: your document has a line-feed at the end of each line. Probably a line feed makes it look like a one-liner from a layout perspective, even though it isn't reset rInf.GetLineStart() to 0? (All assumptions - I have no intimate knowledge of any of this, so everything I say or imply may be incorrect.) My patches were simply based on trial and error - looking for some rational pattern.