example workload that triggers these bottlenecks: mass replace of a character in a large
document
see: https://phabricator.kde.org/D17437
bug report with test case file: https://bugs.kde.org/show_bug.cgi?id=333517
these changes are not sufficient to have it working properly but perf already tells it is much better
| No Linters Available |
| No Unit Test Coverage |
| Buildable 5855 | |
| Build 5873: arc lint + arc unit |
Could you try this patch?
With it, the test case from the bug properly works with < 3GB of memory.
It is still VERY slow, but it doesn't lead to an unusable state.
Before, alone the replacement moving ranges that are used for the hl at the end did take extra 4-5 GB of memory.
Yes, even after this optimization :P 10%
Here is the flat profile, after all this (using the test case from the bug, replacing \t with ; and terminating kwrite afterwards)
86.90% 2.05% kwrite libQt5Core.so.5.11.2 [.] QMetaObject::activate 86.50% 0.00% kwrite libKF5TextEditor.so.5.53.0 [.] KateSearchBar::qt_static_metacall 86.48% 0.00% kwrite libKF5TextEditor.so.5.53.0 [.] KateSearchBar::replaceAll 86.48% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QAbstractButton::clicked 86.48% 0.00% kwrite [unknown] [.] 0x00007ff1fbcecb00 86.08% 0.13% kwrite libKF5TextEditor.so.5.53.0 [.] KateSearchBar::findAll 72.36% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] KateMatch::replace 58.83% 0.08% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::replaceText 30.05% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::removeText 29.33% 0.18% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::editRemoveText 28.24% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::insertText 27.38% 0.14% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::editInsertText 14.03% 0.15% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::removeText 13.01% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::insertText 10.65% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::textRemoved 10.56% 0.76% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextCursor::setPosition 10.24% 0.01% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::textInserted 10.10% 0.02% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::QSlotObject<void (KateVi::NormalViMode::*)(KTextEditor::Document*, KTextEditor::Range), QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void>::impl 10.09% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctionPointer<void (KateVi::NormalViMode::*)(KTextEditor::Document*, KTextEditor::Range)>::call<QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void> 10.05% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1>, QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void, void (KateVi::NormalViMode::*)(KTextEditor::Document*, KTextEditor::Range)>::call 9.04% 1.36% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::removeText 9.04% 0.11% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextCursor::setPosition 8.42% 1.23% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::insertText 8.39% 0.02% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::QSlotObject<void (KTextEditor::DocumentPrivate::*)(KTextEditor::Document*, KTextEditor::Range const&), QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void>::impl 8.38% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctionPointer<void (KTextEditor::DocumentPrivate::*)(KTextEditor::Document*, KTextEditor::Range const&)>::call<QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void> 8.34% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1>, QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void, void (KTextEditor::DocumentPrivate::*)(KTextEditor::Document*, KTextEditor::Range const&)>::call 8.13% 0.25% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::saveEditingPositions 7.63% 0.54% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::Marks::setMark 6.48% 0.40% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextRange::fixLookup 6.45% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] KateMatch::searchText 5.84% 0.19% kwrite libKF5TextEditor.so.5.53.0 [.] KateBuffer::plainLine 5.48% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::newMovingRange 5.48% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::searchText 5.47% 0.20% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::line 5.36% 0.14% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextRange::TextRange 4.99% 0.11% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextRange::checkValidity 4.92% 0.13% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::NormalViMode::textInserted 4.85% 0.12% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::NormalViMode::textRemoved 4.69% 1.65% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::blockForLine 4.04% 1.08% kwrite libKF5TextEditor.so.5.53.0 [.] QBasicAtomicInteger<int>::load 3.89% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::textInserted 3.84% 1.52% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::RefCount::isShared 3.82% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::textRemoved 3.71% 0.13% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::insertCursor 3.66% 0.01% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextRange::~TextRange 3.59% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextRange::~TextRange 3.56% 0.18% kwrite libKF5TextEditor.so.5.53.0 [.] QSet<Kate::TextCursor*>::insert 3.46% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::Marks::setLastChange 3.44% 0.92% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::RefCount::deref 3.35% 2.62% kwrite libKF5TextEditor.so.5.53.0 [.] QAtomicOps<int>::load<int> 3.32% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] KateMatch::buildReplacement 3.15% 0.28% kwrite libKF5TextEditor.so.5.53.0 [.] QHash<Kate::TextCursor*, QHashDummyValue>::insert 3.15% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::removeCursor 3.06% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] QSet<Kate::TextCursor*>::remove 3.02% 0.19% kwrite libKF5TextEditor.so.5.53.0 [.] KatePlainTextSearch::search 2.99% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] KateLayoutCache::qt_static_metacall 2.93% 0.49% kwrite libKF5TextEditor.so.5.53.0 [.] QHash<Kate::TextCursor*, QHashDummyValue>::remove 2.87% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::Marks::setFinishEditYanked 2.85% 0.29% kwrite libKF5TextEditor.so.5.53.0 [.] QSet<Kate::TextCursor*>::const_iterator::operator++ 2.84% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] KateLineLayoutMap::slotEditDone 2.82% 0.53% kwrite libKF5TextEditor.so.5.53.0 [.] QHash<Kate::TextCursor*, QHashDummyValue>::findNode 2.68% 0.13% kwrite libKF5TextEditor.so.5.53.0 [.] KateUndoManager::addUndoItem 2.63% 0.08% kwrite libKF5TextEditor.so.5.53.0 [.] KateUndoManager::slotTextRemoved 2.56% 0.14% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::updateRange 2.47% 0.33% kwrite libKF5TextEditor.so.5.53.0 [.] QBasicAtomicInteger<int>::deref 2.47% 0.48% kwrite libKF5TextEditor.so.5.53.0 [.] QString::~QString 2.39% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::removeRange 2.28% 2.19% kwrite libQt5Core.so.5.11.2 [.] QHashData::nextNode 2.26% 0.23% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::text 2.26% 0.20% kwrite libKF5TextEditor.so.5.53.0 [.] QVector<Kate::TextBlock*>::operator[] 2.21% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] KateUndoManager::slotTextInserted 2.19% 0.26% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::line 2.18% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentCursor::atEndOfDocument 2.07% 0.19% kwrite libKF5TextEditor.so.5.53.0 [.] QVector<Kate::TextBlock*>::data 2.05% 0.31% kwrite libKF5TextEditor.so.5.53.0 [.] QAtomicOps<int>::deref<int> 2.01% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::documentEnd 1.98% 1.89% kwrite libKF5TextEditor.so.5.53.0 [.] QArrayData::data 1.97% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] QStack<QSharedPointer<KTextEditor::MovingCursor> >::pop 1.96% 1.80% kwrite libKF5TextEditor.so.5.53.0 [.] QHash<Kate::TextCursor*, QHashDummyValue>::findNode 1.94% 0.18% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextCursor::TextCursor 1.86% 0.11% kwrite libKF5TextEditor.so.5.53.0 [.] QSet<Kate::TextRange*>::insert 1.86% 1.78% kwrite kwrite [.] std::__atomic_base<int>::operator-- 1.82% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::lineLength 1.80% 0.23% kwrite libKF5TextEditor.so.5.53.0 [.] KateRegExpSearch::buildReplacement 1.71% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] QHash<Kate::TextRange*, QHashDummyValue>::insert 1.70% 0.13% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::blockForIndex 1.65% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::line 1.62% 0.23% kwrite libKF5TextEditor.so.5.53.0 [.] QBasicAtomicInteger<int>::ref 1.58% 0.02% kwrite libKF5TextEditor.so.5.53.0 [.] KateLayoutCache::removeText 1.57% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::kateTextLine 1.54% 0.02% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::Marks::setStartEditYanked 1.52% 1.41% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::startLine 1.48% 1.46% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextCursor::lineInBlock 1.43% 0.28% kwrite libKF5TextEditor.so.5.53.0 [.] KateUndoGroup::addItem 1.43% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] QVector<Kate::TextBlock*>::detach 1.38% 0.20% kwrite libKF5TextEditor.so.5.53.0 [.] QAtomicOps<int>::ref<int> 1.37% 0.02% kwrite libKF5TextEditor.so.5.53.0 [.] KateLayoutCache::insertText 1.32% 0.30% kwrite libKF5TextEditor.so.5.53.0 [.] QTypedArrayData<Kate::TextBlock*>::begin 1.32% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] KateRegExpSearch::escapePlaintext 1.31% 0.28% kwrite libKF5TextEditor.so.5.53.0 [.] QHash<Kate::TextRange*, QHashDummyValue>::findNode 1.31% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KateCompletionWidget::qt_static_metacall 1.30% 1.21% kwrite libc-2.28.so [.] malloc 1.29% 1.14% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::Cursor::line 1.26% 1.21% kwrite kwrite [.] std::__atomic_base<int>::operator++ 1.24% 0.29% kwrite libKF5TextEditor.so.5.53.0 [.] QVector<Kate::TextBlock*>::isDetached 1.23% 0.51% kwrite libKF5TextEditor.so.5.53.0 [.] QVector<Kate::TextBlock*>::operator[] 1.23% 0.12% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::SwapFile::qt_static_metacall 1.22% 0.34% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::QSharedPointer 1.21% 0.00% kwrite [unknown] [.] 0x00005648626a95b0 1.19% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] QSet<Kate::TextRange*>::remove 1.14% 0.11% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::~QSharedPointer 1.13% 0.15% kwrite libKF5TextEditor.so.5.53.0 [.] QHash<Kate::TextRange*, QHashDummyValue>::remove 1.11% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] QVector<QSharedPointer<KTextEditor::MovingCursor> >::resize 1.09% 0.29% kwrite libKF5TextEditor.so.5.53.0 [.] QVector<QSharedPointer<KTextEditor::MovingCursor> >::reallocData 1.08% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::MovingRange::setRange 1.08% 0.23% kwrite libKF5TextEditor.so.5.53.0 [.] QHash<Kate::TextRange*, int>::findNode 1.07% 0.23% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::RefCount::ref 1.05% 0.15% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::deref 1.04% 0.59% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextCursor::line 1.03% 0.32% kwrite libKF5TextEditor.so.5.53.0 [.] QTypedArrayData<Kate::TextBlock*>::data 1.03% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] QStringList::~QStringList 1.00% 0.91% kwrite libKF5TextEditor.so.5.53.0 [.] QHash<Kate::TextRange*, QHashDummyValue>::findNode
Hm indeed. I remember when the patch was added that added the editing positions. At that time I was not convinced about this, and now even performance problems show up.
What I wonder is: Usually, these editing positions should just be the ones the user really jumped to or actively did something. But when you do a search & replace, this should completely be disabled imho... Maybe that would also be a fix to remove this bottleneck entirely for specific use cases. (or remove this feature altogether again :p )
Same issue with the marks of the vimode.
Perhaps they should not triggered at all if vi mode is off ;=)
In any case, this test shows a lot of bottlenecks.
e.g. even the l->toVirtualColumn that is used for block mode did show up like hell because it was accidently done always ;=)
that had a >> 10% usage here for some runs during patching.
:=) at least we now beat atom, with the test case file and replace all \t >= ; it just crashed here after "non responding".
avoid a lot of allocations for the sets
we can just use a vector and remember if we need to update a range, in worst case, twice inserted
only update edit position stack once per transaction with last position of any primitive
New state:
85.24% 0.00% kwrite kwrite [.] main 85.21% 0.00% kwrite libc-2.28.so [.] __libc_start_main 85.19% 0.00% kwrite kwrite [.] _start 84.96% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QApplicationPrivate::notify_helper 84.96% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QApplication::notify 84.95% 0.00% kwrite libQt5Core.so.5.11.2 [.] QCoreApplication::notifyInternal2 84.84% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QWidget::event 84.63% 0.00% kwrite libQt5Core.so.5.11.2 [.] QEventLoop::exec 84.62% 0.00% kwrite libQt5Core.so.5.11.2 [.] QCoreApplication::exec 83.54% 2.38% kwrite libQt5Core.so.5.11.2 [.] QMetaObject::activate 83.43% 0.00% kwrite libQt5Gui.so.5.11.2 [.] QWindowSystemInterface::sendWindowSystemEvents 83.43% 0.00% kwrite libQt5XcbQpa.so.5.11.2 [.] 0x00007fd225febdac 83.43% 0.00% kwrite libQt5Gui.so.5.11.2 [.] QGuiApplicationPrivate::processWindowSystemEvent 83.42% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] 0x00007fd22db585d4 83.42% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QApplicationPrivate::sendMouseEvent 83.42% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] 0x00007fd22db5598b 83.42% 0.00% kwrite libQt5Gui.so.5.11.2 [.] QGuiApplicationPrivate::processMouseEvent 83.42% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QAbstractButton::mouseReleaseEvent 82.87% 0.00% kwrite libKF5TextEditor.so.5.53.0 [.] KateSearchBar::qt_static_metacall 82.87% 0.00% kwrite libKF5TextEditor.so.5.53.0 [.] KateSearchBar::replaceAll 82.87% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QAbstractButton::clicked 82.87% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] 0x00007fd22dbe5a1b 82.87% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] 0x00007fd22dbe6e11 82.81% 0.25% kwrite libKF5TextEditor.so.5.53.0 [.] KateSearchBar::findAll 66.79% 0.24% kwrite libKF5TextEditor.so.5.53.0 [.] KateMatch::replace 59.00% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::replaceText 29.71% 0.22% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::insertText 29.14% 0.12% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::removeText 28.54% 0.18% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::editInsertText 28.09% 0.27% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::editRemoveText 16.95% 0.19% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::removeText 15.63% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::insertText 9.52% 0.82% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::removeText 9.29% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KateMatch::searchText 8.37% 0.24% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::searchText 7.89% 0.83% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::insertText 7.82% 0.28% kwrite libKF5TextEditor.so.5.53.0 [.] KateBuffer::plainLine 7.75% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::textInserted 7.34% 0.27% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::line 7.01% 0.12% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::textInserted 6.25% 0.14% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::textRemoved 5.96% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::QSlotObject<void (KateVi::NormalViMode::*)(KTextEditor::Document*, KTextEditor::Range), QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void>::impl 5.95% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctionPointer<void (KateVi::NormalViMode::*)(KTextEditor::Document*, KTextEditor::Range)>::call<QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void> 5.87% 0.08% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1>, QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void, void (KateVi::NormalViMode::*)(KTextEditor::Document*, KTextEditor::Range)>::call 5.03% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::textRemoved 4.78% 0.13% kwrite libKF5TextEditor.so.5.53.0 [.] KateMatch::buildReplacement 4.57% 1.64% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::blockForLine 4.42% 0.27% kwrite libKF5TextEditor.so.5.53.0 [.] KatePlainTextSearch::search 4.38% 0.33% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::text 4.11% 1.14% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::RefCount::deref 4.11% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] KateUndoManager::addUndoItem 3.97% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] std::vector<Kate::TextRange*, std::allocator<Kate::TextRange*> >::push_back 3.92% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] std::vector<Kate::TextRange*, std::allocator<Kate::TextRange*> >::emplace_back<Kate::TextRange*> 3.81% 0.28% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextRange::checkValidity 3.75% 0.69% kwrite libKF5TextEditor.so.5.53.0 [.] QString::~QString 3.65% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] KateUndoManager::slotTextRemoved 3.60% 0.21% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::NormalViMode::textInserted 3.53% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] KateLayoutCache::qt_static_metacall 3.42% 0.35% kwrite libKF5TextEditor.so.5.53.0 [.] std::vector<Kate::TextRange*, std::allocator<Kate::TextRange*> >::_M_realloc_insert<Kate::TextRange*> 3.41% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] KateLineLayoutMap::slotEditDone 3.40% 0.08% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::SwapFile::qt_static_metacall 3.36% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] KateCompletionWidget::qt_static_metacall 3.16% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KateUndoManager::slotTextInserted 3.14% 0.82% kwrite libKF5TextEditor.so.5.53.0 [.] QBasicAtomicInteger<int>::load 2.93% 0.02% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::QSlotObject<void (KateViewInternal::*)(KTextEditor::Document*, KTextEditor::Range const&), QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void>::impl 2.93% 0.31% kwrite libKF5TextEditor.so.5.53.0 [.] KateRegExpSearch::buildReplacement 2.93% 0.02% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctionPointer<void (KateViewInternal::*)(KTextEditor::Document*, KTextEditor::Range const&)>::call<QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void> 2.92% 0.36% kwrite libKF5TextEditor.so.5.53.0 [.] KateUndoGroup::addItem 2.90% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1>, QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void, void (KateViewInternal::*)(KTextEditor::Document*, KTextEditor::Range const&)>::call 2.87% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] KateViewInternal::documentTextInserted 2.82% 0.80% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::Marks::setMark 2.76% 0.37% kwrite libKF5TextEditor.so.5.53.0 [.] QBasicAtomicInteger<int>::deref 2.73% 0.29% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::line 2.63% 2.26% kwrite libKF5TextEditor.so.5.53.0 [.] QAtomicOps<int>::load<int> 2.54% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentCursor::atEndOfDocument 2.45% 0.26% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextRange::fixLookup 2.41% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::documentEnd 2.41% 0.34% kwrite libKF5TextEditor.so.5.53.0 [.] QAtomicOps<int>::deref<int> 2.30% 0.20% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextRange::setRange 2.27% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::lineLength 2.13% 0.14% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::NormalViMode::textRemoved 2.07% 2.07% kwrite kwrite [.] std::__atomic_base<int>::operator-- 1.98% 0.14% kwrite libKF5TextEditor.so.5.53.0 [.] KateCompletionWidget::insertText 1.98% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::SwapFile::insertText 1.95% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::line 1.92% 0.25% kwrite libKF5TextEditor.so.5.53.0 [.] QBasicAtomicInteger<int>::ref 1.85% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] KateLayoutCache::removeText 1.84% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctionPointer<void (KateViewInternal::*)(KTextEditor::Document*, KTextEditor::Range const&, QString const&)>::call<QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&, QString const&>, void> 1.83% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::QSlotObject<void (KateViewInternal::*)(KTextEditor::Document*, KTextEditor::Range const&, QString const&), QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&, QString const&>, void>::impl 1.82% 0.16% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::~QSharedPointer 1.80% 0.50% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::QSharedPointer 1.78% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::kateTextLine 1.77% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1, 2>, QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&, QString const&>, void, void (KateViewInternal::*)(KTextEditor::Document*, KTextEditor::Range const&, QString const&)>::call 1.76% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] KateRegExpSearch::escapePlaintext 1.75% 0.23% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::deref 1.72% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] KateViewInternal::documentTextRemoved 1.71% 0.24% kwrite libKF5TextEditor.so.5.53.0 [.] QAtomicOps<int>::ref<int> 1.69% 1.69% kwrite libKF5TextEditor.so.5.53.0 [.] QArrayData::data 1.64% 0.02% kwrite libKF5TextEditor.so.5.53.0 [.] KateLayoutCache::insertText 1.63% 0.00% kwrite libc-2.28.so [.] __GI___libc_malloc (inlined) 1.63% 1.63% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::startLine 1.60% 0.11% kwrite libKF5TextEditor.so.5.53.0 [.] std::__uninitialized_move_if_noexcept_a<Kate::TextRange**, Kate::TextRange**, std::allocator<Kate::TextRange*> > 1.56% 0.59% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::RefCount::isShared 1.53% 0.38% kwrite libQt5Core.so.5.11.2 [.] QArrayData::allocate 1.53% 0.34% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::RefCount::ref 1.52% 0.19% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::deref 1.51% 0.62% kwrite libQt5Core.so.5.11.2 [.] QIODevice::write 1.47% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] KateRegExpSearch::buildReplacement 1.47% 1.47% kwrite kwrite [.] std::__atomic_base<int>::operator++ 1.47% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::Marks::setLastChange 1.43% 0.27% kwrite libstdc++.so.6.0.25 [.] __dynamic_cast 1.36% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::SwapFile::removeText 1.35% 0.16% kwrite libKF5TextEditor.so.5.53.0 [.] QVector<KTextEditor::Range>::append 1.34% 0.22% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::ref 1.32% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] KateCompletionWidget::removeText 1.31% 0.82% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextCursor::line 1.31% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QMainWindow::event 1.31% 0.00% kwrite libKF5XmlGui.so.5.53.0 [.] KMainWindow::event 1.31% 0.17% kwrite libQt5Core.so.5.11.2 [.] QDataStream::operator<< 1.31% 0.33% kwrite libKF5TextEditor.so.5.53.0 [.] QString::QString 1.29% 1.29% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::Cursor::line 1.26% 0.00% kwrite libKF5XmlGui.so.5.53.0 [.] KXmlGuiWindow::event 1.24% 0.73% kwrite libQt5Gui.so.5.11.2 [.] QAccessible::queryAccessibleInterface 1.23% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] KateModifiedRemoveText::KateModifiedRemoveText 1.23% 0.12% kwrite libKF5TextEditor.so.5.53.0 [.] qDeleteAll<QList<KateUndoGroup*> > 1.22% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] KateModifiedInsertText::KateModifiedInsertText 1.21% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] QStringList::~QStringList 1.21% 0.00% kwrite libglib-2.0.so.0.5800.1 [.] g_main_context_dispatch 1.21% 0.00% kwrite libQt5Core.so.5.11.2 [.] QEventDispatcherGlib::processEvents 1.21% 0.00% kwrite libglib-2.0.so.0.5800.1 [.] g_main_context_iteration 1.21% 0.00% kwrite libglib-2.0.so.0.5800.1 [.] 0x00007fd22a5bdf88 1.20% 0.00% kwrite libQt5XcbQpa.so.5.11.2 [.] 0x00007fd225febda1 1.20% 0.08% kwrite libKF5TextEditor.so.5.53.0 [.] std::__uninitialized_copy_a<std::move_iterator<Kate::TextRange**>, Kate::TextRange**, Kate::TextRange*> 1.20% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::plainKateTextLine 1.20% 0.18% kwrite libstdc++.so.6.0.25 [.] __cxxabiv1::__si_class_type_info::__do_dyncast 1.17% 0.21% kwrite libKF5TextEditor.so.5.53.0 [.] QList<QString>::~QList 1.15% 0.20% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::operator< 1.15% 0.14% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextHistory::addEntry 1.14% 0.11% kwrite libKF5TextEditor.so.5.53.0 [.] QSet<Kate::TextCursor*>::const_iterator::operator++ 1.14% 0.49% kwrite libKF5TextEditor.so.5.53.0 [.] QVector<Kate::TextBlock*>::operator[] 1.14% 0.08% kwrite libKF5TextEditor.so.5.53.0 [.] std::uninitialized_copy<std::move_iterator<Kate::TextRange**>, Kate::TextRange**> 1.14% 1.14% kwrite libc-2.28.so [.] malloc 1.14% 0.24% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::Range::Range 1.14% 0.03% kwrite libQt5Core.so.5.11.2 [.] QTimer::start 1.12% 0.13% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::updateRange 1.12% 0.14% kwrite libQt5Core.so.5.11.2 [.] QObject::startTimer 1.11% 0.00% kwrite [unknown] [.] 0xffffffffffffffff 1.10% 0.00% kwrite libQt5Core.so.5.11.2 [.] QObject::event 1.10% 0.00% kwrite libQt5Core.so.5.11.2 [.] QCoreApplicationPrivate::sendPostedEvents 1.09% 0.29% kwrite libKF5TextEditor.so.5.53.0 [.] QTypedArrayData<Kate::TextBlock*>::begin 1.09% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::qMakeForeachContainer<QSet<Kate::TextCursor*>&> 1.09% 0.00% kwrite libQt5Core.so.5.11.2 [.] 0x00007fd22d1459d3 1.09% 0.32% kwrite libQt5Core.so.5.11.2 [.] QFileDevice::writeData 1.08% 0.00% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::openFile 1.07% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] std::__uninitialized_copy<true>::__uninit_copy<std::move_iterator<Kate::TextRange**>, Kate::TextRange**> 1.07% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] QHash<Kate::TextCursor*, QHashDummyValue>::const_iterator::operator++ 1.06% 0.00% kwrite libKF5TextEditor.so.5.53.0 [.] KateBuffer::openFile 1.05% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::QForeachContainer<QSet<Kate::TextCursor*> >::QForeachContainer 1.05% 0.00% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::load 1.04% 0.26% kwrite libQt5Core.so.5.11.2 [.] QString::mid 1.04% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::MovingRange::setRange 1.03% 0.14% kwrite libKF5TextEditor.so.5.53.0 [.] QMap<QChar, KTextEditor::MovingCursor*>::value 1.03% 1.03% kwrite libc-2.28.so [.] __strcmp_avx2 1.01% 0.18% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextLoader::readLine 1.00% 0.27% kwrite libQt5Core.so.5.11.2 [.] QObject::killTimer
avoid costly lineLength computation
this really is expensive, as we do a lookup of the line that kills
the cache in the block lookup + we can skip that easily for non-last lines
costs 2% for large replaces
2.54% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentCursor::atEndOfDocument
:P ROFL
85.51% 0.00% kwrite kwrite [.] main 85.49% 0.00% kwrite libc-2.28.so [.] __libc_start_main 85.47% 0.00% kwrite kwrite [.] _start 85.09% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QApplicationPrivate::notify_helper 85.09% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QApplication::notify 85.08% 0.00% kwrite libQt5Core.so.5.11.2 [.] QCoreApplication::notifyInternal2 84.95% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QWidget::event 84.82% 0.00% kwrite libQt5Core.so.5.11.2 [.] QEventLoop::exec 84.81% 0.00% kwrite libQt5Core.so.5.11.2 [.] QCoreApplication::exec 83.75% 2.45% kwrite libQt5Core.so.5.11.2 [.] QMetaObject::activate 83.66% 0.00% kwrite libQt5Gui.so.5.11.2 [.] QGuiApplicationPrivate::processWindowSystemEvent 83.66% 0.00% kwrite libQt5Gui.so.5.11.2 [.] QWindowSystemInterface::sendWindowSystemEvents 83.66% 0.00% kwrite libQt5XcbQpa.so.5.11.2 [.] 0x00007f8e008eadac 83.64% 0.00% kwrite libQt5Gui.so.5.11.2 [.] QGuiApplicationPrivate::processMouseEvent 83.64% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] 0x00007f8e084575d4 83.64% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QApplicationPrivate::sendMouseEvent 83.64% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] 0x00007f8e0845498b 83.63% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QAbstractButton::mouseReleaseEvent 83.01% 0.00% kwrite libKF5TextEditor.so.5.53.0 [.] KateSearchBar::qt_static_metacall 83.01% 0.00% kwrite libKF5TextEditor.so.5.53.0 [.] KateSearchBar::replaceAll 83.01% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QAbstractButton::clicked 83.01% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] 0x00007f8e084e4a1b 83.01% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] 0x00007f8e084e5e11 82.95% 0.28% kwrite libKF5TextEditor.so.5.53.0 [.] KateSearchBar::findAll 69.05% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] KateMatch::replace 65.42% 0.12% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::replaceText 34.14% 0.19% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::insertText 32.93% 0.22% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::editInsertText 31.05% 0.18% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::removeText 29.85% 0.32% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::editRemoveText 17.63% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::removeText 17.45% 0.22% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::insertText 10.64% 0.87% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::removeText 9.25% 0.02% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::textInserted 9.20% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] KateMatch::searchText 8.96% 0.98% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::insertText 7.97% 0.11% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::searchText 7.65% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::textInserted 6.54% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::QSlotObject<void (KateVi::NormalViMode::*)(KTextEditor::Document*, KTextEditor::Range), QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void>::impl 6.54% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctionPointer<void (KateVi::NormalViMode::*)(KTextEditor::Document*, KTextEditor::Range)>::call<QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void> 6.46% 0.08% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1>, QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void, void (KateVi::NormalViMode::*)(KTextEditor::Document*, KTextEditor::Range)>::call 5.81% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::textRemoved 5.12% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::textRemoved 4.92% 0.29% kwrite libKF5TextEditor.so.5.53.0 [.] KateBuffer::plainLine 4.88% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] KateUndoManager::addUndoItem 4.41% 0.26% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::line 4.25% 0.24% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextRange::checkValidity 4.18% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] KateUndoManager::slotTextRemoved 4.17% 0.24% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::NormalViMode::textInserted 4.13% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] std::vector<Kate::TextRange*, std::allocator<Kate::TextRange*> >::push_back 4.06% 0.11% kwrite libKF5TextEditor.so.5.53.0 [.] std::vector<Kate::TextRange*, std::allocator<Kate::TextRange*> >::emplace_back<Kate::TextRange*> 4.02% 0.36% kwrite libKF5TextEditor.so.5.53.0 [.] KatePlainTextSearch::search 3.92% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] KateLayoutCache::qt_static_metacall 3.80% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KateUndoManager::slotTextInserted 3.69% 0.26% kwrite libKF5TextEditor.so.5.53.0 [.] KateLineLayoutMap::slotEditDone 3.66% 0.34% kwrite libKF5TextEditor.so.5.53.0 [.] std::vector<Kate::TextRange*, std::allocator<Kate::TextRange*> >::_M_realloc_insert<Kate::TextRange*> 3.63% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::QSlotObject<void (KateViewInternal::*)(KTextEditor::Document*, KTextEditor::Range const&), QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void>::impl 3.61% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctionPointer<void (KateViewInternal::*)(KTextEditor::Document*, KTextEditor::Range const&)>::call<QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void> 3.59% 0.21% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::SwapFile::qt_static_metacall 3.56% 0.04% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1>, QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&>, void, void (KateViewInternal::*)(KTextEditor::Document*, KTextEditor::Range const&)>::call 3.55% 0.06% kwrite libKF5TextEditor.so.5.53.0 [.] KateViewInternal::documentTextInserted 3.55% 0.91% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::RefCount::deref 3.51% 0.40% kwrite libKF5TextEditor.so.5.53.0 [.] KateUndoGroup::addItem 3.39% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] KateCompletionWidget::qt_static_metacall 3.33% 0.25% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::text 3.24% 0.49% kwrite libKF5TextEditor.so.5.53.0 [.] QString::~QString 3.01% 0.86% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::Marks::setMark 2.84% 0.84% kwrite libKF5TextEditor.so.5.53.0 [.] QBasicAtomicInteger<int>::load 2.77% 0.24% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextRange::fixLookup 2.52% 0.38% kwrite libKF5TextEditor.so.5.53.0 [.] QBasicAtomicInteger<int>::deref 2.41% 0.16% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextRange::setRange 2.33% 2.01% kwrite libKF5TextEditor.so.5.53.0 [.] QAtomicOps<int>::load<int> 2.32% 0.81% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::blockForLine 2.29% 0.25% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::line 2.19% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::SwapFile::insertText 2.18% 0.20% kwrite libKF5TextEditor.so.5.53.0 [.] KateCompletionWidget::insertText 2.16% 0.33% kwrite libKF5TextEditor.so.5.53.0 [.] QAtomicOps<int>::deref<int> 2.14% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::NormalViMode::textRemoved 2.09% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::kateTextLine 2.01% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] KateLayoutCache::removeText 1.97% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] KateRegExpSearch::escapePlaintext 1.87% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KateLayoutCache::insertText 1.87% 1.87% kwrite kwrite [.] std::__atomic_base<int>::operator-- 1.82% 0.02% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::QSlotObject<void (KateViewInternal::*)(KTextEditor::Document*, KTextEditor::Range const&, QString const&), QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&, QString const&>, void>::impl 1.81% 0.32% kwrite libstdc++.so.6.0.25 [.] __dynamic_cast 1.81% 0.02% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctionPointer<void (KateViewInternal::*)(KTextEditor::Document*, KTextEditor::Range const&, QString const&)>::call<QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&, QString const&>, void> 1.80% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1, 2>, QtPrivate::List<KTextEditor::Document*, KTextEditor::Range const&, QString const&>, void, void (KateViewInternal::*)(KTextEditor::Document*, KTextEditor::Range const&, QString const&)>::call 1.76% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KateViewInternal::documentTextRemoved 1.71% 0.21% kwrite libKF5TextEditor.so.5.53.0 [.] KateRegExpSearch::buildReplacement 1.70% 0.13% kwrite libKF5TextEditor.so.5.53.0 [.] std::__uninitialized_move_if_noexcept_a<Kate::TextRange**, Kate::TextRange**, std::allocator<Kate::TextRange*> > 1.64% 0.12% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::~QSharedPointer 1.64% 0.00% kwrite libc-2.28.so [.] __GI___libc_malloc (inlined) 1.62% 0.27% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::deref 1.58% 0.21% kwrite libKF5TextEditor.so.5.53.0 [.] QBasicAtomicInteger<int>::ref 1.56% 0.64% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::RefCount::isShared 1.55% 0.23% kwrite libstdc++.so.6.0.25 [.] __cxxabiv1::__si_class_type_info::__do_dyncast 1.51% 0.87% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextCursor::line 1.49% 1.49% kwrite libKF5TextEditor.so.5.53.0 [.] QArrayData::data 1.48% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] KateModifiedRemoveText::KateModifiedRemoveText 1.47% 1.47% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::Cursor::line 1.47% 0.43% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::QSharedPointer 1.46% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::Marks::setLastChange 1.43% 0.36% kwrite libQt5Core.so.5.11.2 [.] QArrayData::allocate 1.42% 0.22% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextHistory::addEntry 1.42% 0.33% kwrite libQt5Core.so.5.11.2 [.] QIODevice::write 1.41% 0.22% kwrite libKF5TextEditor.so.5.53.0 [.] QAtomicOps<int>::ref<int> 1.40% 0.83% kwrite libQt5Gui.so.5.11.2 [.] QAccessible::queryAccessibleInterface 1.40% 0.34% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::RefCount::ref 1.35% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] qDeleteAll<QList<KateUndoGroup*> > 1.35% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::deref 1.34% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KateModifiedInsertText::KateModifiedInsertText 1.34% 0.32% kwrite libKF5TextEditor.so.5.53.0 [.] QString::QString 1.33% 1.33% kwrite libc-2.28.so [.] __strcmp_avx2 1.33% 0.24% kwrite libKF5TextEditor.so.5.53.0 [.] QVector<KTextEditor::Range>::append 1.32% 1.32% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::startLine 1.30% 0.24% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::operator< 1.30% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::plainKateTextLine 1.25% 0.27% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::Range::Range 1.25% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] std::__uninitialized_copy_a<std::move_iterator<Kate::TextRange**>, Kate::TextRange**, Kate::TextRange*> 1.24% 0.26% kwrite libQt5Gui.so.5.11.2 [.] QAccessibleEvent::accessibleInterface 1.23% 0.10% kwrite libKF5TextEditor.so.5.53.0 [.] QSet<Kate::TextCursor*>::const_iterator::operator++ 1.23% 0.30% kwrite libQt5Core.so.5.11.2 [.] QFileDevice::writeData 1.23% 0.10% kwrite libQt5Core.so.5.11.2 [.] QDataStream::operator<< 1.23% 1.23% kwrite libc-2.28.so [.] malloc 1.23% 0.04% kwrite libQt5Core.so.5.11.2 [.] QTimer::start 1.23% 0.00% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::openFile 1.22% 0.00% kwrite libKF5XmlGui.so.5.53.0 [.] KMainWindow::event 1.22% 0.00% kwrite libQt5Widgets.so.5.11.2 [.] QMainWindow::event 1.22% 0.16% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBlock::updateRange 1.21% 0.17% kwrite libQt5Core.so.5.11.2 [.] QObject::startTimer 1.20% 0.00% kwrite libKF5TextEditor.so.5.53.0 [.] KateBuffer::openFile 1.19% 1.19% kwrite kwrite [.] std::__atomic_base<int>::operator++ 1.18% 0.00% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextBuffer::load 1.17% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] std::uninitialized_copy<std::move_iterator<Kate::TextRange**>, Kate::TextRange**> 1.17% 0.00% kwrite libKF5XmlGui.so.5.53.0 [.] KXmlGuiWindow::event 1.16% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::SwapFile::removeText 1.16% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::MovingRange::setRange 1.16% 0.15% kwrite libKF5TextEditor.so.5.53.0 [.] QHash<Kate::TextCursor*, QHashDummyValue>::const_iterator::operator++ 1.16% 0.00% kwrite libQt5Core.so.5.11.2 [.] QEventDispatcherGlib::processEvents 1.16% 0.00% kwrite libQt5XcbQpa.so.5.11.2 [.] 0x00007f8e008eada1 1.16% 0.00% kwrite libglib-2.0.so.0.5800.1 [.] g_main_context_iteration 1.16% 0.00% kwrite libglib-2.0.so.0.5800.1 [.] g_main_context_dispatch 1.16% 0.00% kwrite libglib-2.0.so.0.5800.1 [.] 0x00007f8e04ebcf88 1.16% 0.14% kwrite libKF5TextEditor.so.5.53.0 [.] QMap<QChar, KTextEditor::MovingCursor*>::value 1.15% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] Kate::TextLoader::readLine 1.14% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] KateCompletionWidget::removeText 1.13% 0.18% kwrite libKF5TextEditor.so.5.53.0 [.] KateEditInsertTextUndo::mergeWith 1.13% 0.00% kwrite [unknown] [.] 0xffffffffffffffff 1.12% 0.27% kwrite libQt5Core.so.5.11.2 [.] QString::mid 1.11% 0.05% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::qMakeForeachContainer<QSet<Kate::TextCursor*>&> 1.11% 0.09% kwrite libKF5TextEditor.so.5.53.0 [.] std::__uninitialized_copy<true>::__uninit_copy<std::move_iterator<Kate::TextRange**>, Kate::TextRange**> 1.10% 0.07% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::DocumentPrivate::line 1.10% 0.48% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::operator>= 1.08% 0.23% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::Range::Range 1.08% 0.12% kwrite libKF5TextEditor.so.5.53.0 [.] QVector<QPair<int, QExplicitlySharedDataPointer<KateLineLayout> > >::end 1.07% 0.17% kwrite libKF5TextEditor.so.5.53.0 [.] QVector<QPair<int, QExplicitlySharedDataPointer<KateLineLayout> > >::detach 1.07% 0.00% kwrite libQt5Core.so.5.11.2 [.] QObject::event 1.06% 0.18% kwrite libKF5TextEditor.so.5.53.0 [.] QSharedPointer<Kate::TextLineData>::ref 1.05% 0.00% kwrite libQt5Core.so.5.11.2 [.] QCoreApplicationPrivate::sendPostedEvents 1.05% 0.19% kwrite libKF5TextEditor.so.5.53.0 [.] QtPrivate::QForeachContainer<QSet<Kate::TextCursor*> >::QForeachContainer 1.04% 0.28% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::MovingRange::toRange 1.04% 0.03% kwrite libKF5TextEditor.so.5.53.0 [.] KateVi::Marks::setFinishEditYanked 1.04% 0.00% kwrite libQt5Core.so.5.11.2 [.] 0x00007f8e07a449d3 1.03% 0.22% kwrite libKF5TextEditor.so.5.53.0 [.] QMapData<QChar, KTextEditor::MovingCursor*>::findNode 1.03% 1.03% kwrite libQt5Core.so.5.11.2 [.] QRingBuffer::reserve 1.00% 0.62% kwrite libKF5TextEditor.so.5.53.0 [.] KTextEditor::MovingCursor::toCursor
I have tried this one. Yes, works much more better!
Nice to see that my own (unloved) patch wake your interest to dig into that issue :-)
As usual understand I most of your talk here not really, and these perf protocols give me no light on the first sight. But thank you that you let me participate.
I think with this patch need KateSearchBar still some improvement. Like to give a hint that S&R is running and the possibility to abort. Would be good to add some test if a S&R may run out of memory. Not everyone has a lot of RAM available.
Um, just a stupid question: Is there a need to do the S&R just as it is done currently? So far I understand that, is the only reason why that is so f**antastic slow caused by tons off calls which are intend to track a normal editing.
Why not simple do the S&R on the raw file, make a diff, highlight the changes and we are done(?)
I did the tab->semicolon S&R by sed, just for interest, and it takes 2-3 seconds :-/
| src/buffer/katetextrange.h | ||
|---|---|---|
| 367 | I would prefer an additional inline void setValidityCheckRequired(); inline bool validityCheckRequired() const; We don't need a bool in the setter, since we always only mark dirty. And with these accessors, you could add a unit test (in dact, I would like to see one for this). And instead of 'double' I suggest to use the term 'multiple' in the comment. | |
| src/search/katesearchbar.cpp | ||
| 815 | If it's not an issue at all, I prefer to have limits that are realistic. If it scales well for, say 4096, then 4096 is the better choice. | |
Will improve further ;=)
| src/buffer/katetextblock.cpp | ||
|---|---|---|
| 152 | Actually no, that will be again a lot more expensive. | |
| src/buffer/katetextrange.h | ||
| 367 | That makes sense, will change that. | |
| src/search/katesearchbar.cpp | ||
| 815 | Actually, on a decent machine, 64k things work. | |
For the unit test: we have tests for ranges, it makes no sense to test if the bool is set.
As long as the range tests still work, the updates are ok.
:=) Nice that it helps a bit for the speed isue.
As usual understand I most of your talk here not really, and these perf protocols give me no light on the first sight. But thank you that you let me participate.
perf helps you to easily see the hotspots.
e.g. you run
perf report --call-graph dwarf -F 333 kwrite the_test_file...
do your actions there (here replace all).
Afterwards (or if you want only partial trace, press once CTRL-C), you run for the flat profile
perf report --max-stack 1024 --stdio -g none > report.txt
Then you see directly which functions might need attentions.
If e.g. a function takes 10% cumulative, 10% is spend in it and all stuff it calls.
If you see there strange stuff, like e.g. the saveEditingPositons, that is a nice start for optimizations.
perf report
will allow to have some text ui, or use hotspot for graphical bliss. (or generate flamegraph svgs)
- Now, just find all of a char in that fat file is done very quickly
- Doing these tab->semicolon S&R still takes quite a while but works here. System monitor told 1.8G memory was eaten
I think with this patch need KateSearchBar still some improvement. Like to give a hint that S&R is running and the possibility to abort. Would be good to add some test if a S&R may run out of memory. Not everyone has a lot of RAM available.
Um, just a stupid question: Is there a need to do the S&R just as it is done currently? So far I understand that, is the only reason why that is so f**antastic slow caused by tons off calls which are intend to track a normal editing.
Why not simple do the S&R on the raw file, make a diff, highlight the changes and we are done(?) I did the tab->semicolon S&R by sed, just for interest, and it takes 2-3 seconds :-/
The thing is: is you do that this way, you loose all convenience.
e.g. if the user has marks on lines, they will not move, if there are additional highlighings, they are lost, even if you just replace a few letters somewhere.
All that overhead is there for a reason to have nifty features, if you want raw speed, yes, a script or plain command line tool like sed will always win.
But I think we should try to shave of as much overhead as possible, without crippling the user experience.
next minor bottleneck: QList with things a bit larger than a pointer
in addition, here zero sharing is needed or wanted, we can use a plain vector
:P Crash for the win.
| src/buffer/katetextblock.cpp | ||
|---|---|---|
| 290 | This loop now crash in the unit test ./bin/bug313759 )It works again, if I use a QSet, at that one location. | |
Ok, the optimization with the vector fails due delets inside checkValidity(), need to revert that and update docs.
| src/buffer/katetextblock.cpp | ||
|---|---|---|
| 290 | Ok, doesn't work due to the issue that we might delete the range via notifier in checkValidity() :P | |
go back to QSet for the checkValidity, mark in the comments that ranges might be deleted
| src/buffer/katetextblock.cpp | ||
|---|---|---|
| 290 | Noob question: Would QPointer help? if (range && range->isValidityCheckRequired()) { | |
One issue I have with the auto-tests:
With and without any of these changes here locally the vimode_keys test fails.
FAIL! : KeysTest::MacroTests() Compared values are not the same
Actual (kate_document->text()): "completionMacro completionRepeatLastChange completionRep" Expected (expected_text) : "completionMacro completionRepeatLastChange completionRepeatLastChange" Loc: [/home/cullmann/kde/src/frameworks/ktexteditor/autotests/src/vimode/keys.cpp(1512)]
| src/buffer/katetextblock.cpp | ||
|---|---|---|
| 290 | As it is no QObject, no ;=) Actually, I think all workaround might be even more expensive than the QSet. | |
Hmpf, I amended my commit message to contain a CHANGELOG... and stuff bug unfortunately arc land ate that again :P
Hmm, the keys test seem to fail not for all Qt versions.
Qt 5.9 ok
Qt 5.11 bad
see e.g. CI.
Before and after this commit...
This needs now some user testing ;=)
Done as previous, with that testfile. Seams to work so far. Find is still quick, these TAB S&R takes around 2min.
The only curiosity I noticed is that no highliting take effect when there are too much hits(?) Search some number like 5 "142926 matches found" no highlight. Char A "10565 matches found" is highlighted
The only curiosity I noticed is that no highliting take effect when there are too much hits(?) Search some number like 5 "142926 matches found" no highlight. Char A "10565 matches found" is highlighted
Yes, there is now a limit for how many highlightings are done.
If its too much, we skip that, else the rendering will break everthing down ;=)
Yes, there is now a limit for how many highlightings are done.
If its too much, we skip that, else the rendering will break everthing down ;=)
Doing these highliting "on the fly" (only what is/became visible) is probably not possible(?)
BTW: Thanks a lot for your detailed perf explanations! Unfortunately didn't see I more clear, these percent data are not very self explanatory
In principle yes, but I think that is too much work for the case that "mostly all things" are marked ;=)
BTW: Thanks a lot for your detailed perf explanations! Unfortunately didn't see I more clear, these percent data are not very self explanatory
perf is really great, without any large overhead one can directly see (in most cases) were to put effort into.
further info about the flamegraph I attached as SVG here, see on
http://www.brendangregg.com/flamegraphs.html
:=) The run perf on the production servers of netflix, not bad.
Fixed my QSet issues, I was just stupid, I shall have guarded the push_back with the bool, not the later use. ;=)
I guess it's ok to have this in. But please use this version at your daily work ;) I think we don't have many testers...
I always use Kate from git master. Is there anything in particular I should keep an eye out for?
