Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add_regions causes Sublime Text to hang for ever after eating 1.81GB of RAM #1846

Closed
evandrocoan opened this issue Jul 27, 2017 · 6 comments
Closed

Comments

@evandrocoan
Copy link

evandrocoan commented Jul 27, 2017

Summary

After a few hours working over it, I figured out why it is crashing after typing text into Sublime Text. What did I do to figure it out? First I had to find out a automated way to make Sublime Text crash. While walking I remembered about automatic typers, i.e., computer programs which type text on the computer. Then voila, I figured out how to make Sublime Text crash while type text for few hours.

Now I need find out if was the Sublime Text core or some package doing some weird stuff. In the end, after find out exactly the code lines doing such, I am not sure whether it is the packages fault, or the Sublime Text's core. Here I listed all Sublime Text packages I got installed:

"A File Icon",
"Active View Jump Back",
"Add Folder To Project",
"Advanced CSV",
"AdvancedNewFile",
"Alignment",
"AlignTab",
"All Autocomplete",
"amxmodx",
"Amxx Pawn",
"Anaconda",
"ANSIescape",
"ApplySyntax",
"Auto Refresh",
"AutoFileName",
"BBCode",
"Better CoffeeScript",
"BetterFindBuffer",
"BracketHighlighter",
"BufferScroll",
"C++ Completions",
"C++ Snippets",
"C++ Starting Kit",
"Case Conversion",
"Chain of Command",
"ChangeQuotes",
"ChannelRepositoryTools",
"Clear Cursors Carets",
"Clickable URLs",
"Clipboard Scope Copy",
"Color Highlighter",
"ColorHelper",
"ColorSchemeEditor",
"Column Select",
"Compare Side-By-Side",
"ConvertToUTF8",
"Copy Filepath With Line Numbers",
"CopyWithLineNumbersReloaded",
"Default Syntax",
"Delete Current File",
"DictionaryAutoComplete",
"DistractionFreeWindow",
"DocBlockr",
"Dotfiles Syntax Highlighting",
"DuplicateSelections",
"Edit Preferences",
"Emmet",
"ExportHtml",
"ExtendedTabSwitcher",
"Extract Text to File",
"File History",
"File Rename",
"FileDiffs",
"FileManager",
"Find++",
"FindKeyConflicts",
"Fix Project Switch Restart Bug",
"Fix Selection After Indent",
"Fixed Selections Clear",
"Force Rewrite Sublime Settings",
"FuzzyFileNav",
"FuzzyFilePath",
"Generic Config",
"Gist",
"Git",
"GitGutter",
"GitHub Markdown Snippets",
"Glue",
"Google Spell Check",
"GotoLastEditEnhanced",
"Highlight Build Errors",
"HighlightWords",
"Horizontal Scroll",
"Hungry Backspace",
"Increment Selection",
"Incrementor",
"Indent and braces",
"Indent Size",
"Invert Selection",
"Javatar",
"Jedi - Python autocompletion",
"LaTeX Word Count",
"LaTeX-cwl",
"LaTeXTools",
"LESS",
"Line Endings Unify",
"Local History",
"Markdown Preview",
"Markdown to BB Code Converter",
"MarkdownLight",
"Matlab Completions",
"MatlabFilenameAutoComplete",
"MaxPane",
"MoveText",
"MultiEditUtils",
"MultiLingual Dictionary",
"MySQL Snippets",
"Non Text Files",
"Notepad++ Color Scheme",
"Number King",
"Octave",
"Open Auto Completion",
"Origami",
"Override Unpacked Packages",
"OverrideAudit",
"Overwrite Commit Completion",
"Package Control",
"PackageDev",
"PackageResourceViewer",
"Path Translator",
"PostScript",
"PowerCursors",
"Preferences Editor",
"Project Specific Syntax Settings",
"Qt Completions for C++",
"Random Everything",
"RecentActiveFiles",
"ReIndent",
"Rich Plain Text",
"ScopeAlways",
"ScopeHunter",
"Select All Spelling Errors",
"Shell Script",
"Side-by-Side Settings",
"SideBarEnhancements",
"SQL Keyword Uppercase",
"SQLExec",
"SQLTools",
"StickySearch",
"Sublime Text Studio",
"Sublime Tutor",
"sublime-text-2-buildview",
"SublimeCodeIntel",
"SublimeREPL",
"Swift",
"Sync View Scroll",
"SyncedSideBar",
"Syntax Highlighting for Sass",
"SyntaxManager",
"TabsExtra",
"Terminal",
"Text Pastry",
"Toggle",
"TOML",
"Toolbar",
"Trimmer",
"Uncrustify",
"UnitTesting",
"VBScript",
"View Settings Freely",
"Whitespace",
"WordHighlight",
"Wrap Plus",
"x86 and x86_64 Assembly",
"zzz A File Icon zzz"


After gathering these data, I followed the procedures, following with a recorded video to register the behavior for future references, while I tested how Sublime Text would behavior to the stress test:

  • The test computer characteristics are:

    1. Processor Intel i3-390M @2.67 GHz: (Underclocked to 1 GHz)

      C:\>echo %PROCESSOR_ARCHITECTURE% %PROCESSOR_IDENTIFIER% %PROCESSOR_LEVEL% %PROCESSOR_REVISION%
      AMD64 Intel64 Family 6 Model 37 Stepping 5, GenuineIntel 6 2505
      

      image

    2. RAM 8 GB DDR3 @1067 MHz:

      C:\>wmic MemoryChip get BankLabel, Capacity, MemoryType, TypeDetail, Speed
      BankLabel  Capacity    MemoryType  Speed  TypeDetail
      BANK 0     4294967296  24          1067   128
      BANK 2     4294967296  24          1067   128
      

      image

    3. GPU Intel(R) HD Graphics 64 MB of RAM

      C:\>wmic path win32_VideoController get name
      Name
      Intel(R) HD Graphics
      

      image


But before start, I created a complete Sublime Text vanilla install, (except for the settings: "font_size": 11, "highlight_line": true,). It keeps a correct behavior, with no memory increases or crashes. Within a the memory usage of 23 MB all runtime along:
* https://vid.me/2ISqa (Sublime Text Vanilla Test)

  1. First I put all the packages listed above on the "ignored_packages" list:
    * https://vid.me/G2SZb (Sublime Text Disabled Packages Test)
    As may be noticed, Sublime Text keeps the same behavior as on the vanilla install, except for the initial memory amount usage, which is 48 MB.

  2. Next I put 28 packages on the "ignored_packages" list, while keeping all other 127 running free:
    * https://vid.me/mW6Ky (Sublime Text Packages Crashing 1)
    Now, Sublime Text starts leaking memory after 1 minute of the video running. When it got on the minute 6:17, Sublime Text stops responding and starts using the kernel Time, instead of the user time:
    image
    And crashes on the minute 7:15 when I tried to interact with it, generating a dump file with size 0. Worth notice, Sublime Text only crashed when forced a interaction, by clicking on its window. If I did not attempted it, Sublime Text would still be running unresponsive, on kernel time, but not crashing.

  3. As last time, but now there are only 63 packages enabled and 92 packages on the "ignored_packages" list:
    * https://vid.me/A6702 (Sublime Text Packages Crashing 2)
    Sublime Text crashes again, now only more 63 packages to go on and disable.

  4. Now there are only 31 packages enabled and 124 packages on the "ignored_packages" list:
    * https://vid.me/WY6cx (Sublime Text Packages Crashing 3)
    Sublime Text crashes again, now only more 31 packages to go on and disable.

  5. Now there are only 15 packages enabled and 140 packages on the "ignored_packages" list:
    * https://vid.me/zufZp (Sublime Text Packages Crashing 4)
    Wow, now Sublime Text does not crashes! Meaning, one of the 16 packages I just disabled was bugging Sublime Text. They were:

    "Force Rewrite Sublime Settings",
    "FuzzyFileNav",
    "FuzzyFilePath",
    "Generic Config",
    "Gist",
    "Git",
    "GitGutter",
    "GitHub Markdown Snippets",
    "Glue",
    "Google Spell Check",
    "GotoLastEditEnhanced",
    "Highlight Build Errors",
    "HighlightWords",
    "Horizontal Scroll",
    "Hungry Backspace",
    "Increment Selection",
    

That is the end of our journey. After a few making off tests more, I finally nailed it down to the package:

  1. https://github.com/shagabutdinov/sublime-goto-last-edit-enhanced @shagabutdinov "GotoLastEditEnhanced"

Expected behavior

There are a few mutual-exclusive expectations which came from interpreting what the "GotoLastEditEnhanced" package is actually doing:

  1. If Sublime Text is leaking memory, it expected to not leak memory and crash. Related to: Several crashes while typing text #1832 (Several crashes while typing text).
  2. If the package "GotoLastEditEnhanced" is leaking memory, not let the package leak so much memory and make Sublime Text to crash from it. Related to: Packages are allowed to hang Sublime Text Indefinitely #1463 (Packages are allowed to hang Sublime Text Indefinitely).

Actual behavior

As long as more characters are inserted, the Sublime Text RAM usage increases, where it completely stops responding to user input, and start using only CPU kernel time. See https://vid.me/WY6cx (Sublime Text Packages Crashing 3). On this animation, we see my Sublime Text having it RAM usage increases:

slowing_bug

Code used

Open Sublime Text console and run the command:

sublime.active_window().active_view().run_command( "bug" )

import sublime
import sublime_plugin

import threading

class History():
    def __init__(self):
        self.index = 0
        self.max = 0

    def increment(self):
        self.max += 1
        self.index = self.max

class Collection():
    def __init__(self):
        self.list = {}
        self.index = 0

    def get(self, view):
        id = view.id()
        if id not in self.list:
            self.list[id] = History()

        return self.list[id]

collection = Collection()

class Listener(sublime_plugin.EventListener):

    def on_modified(self, view):
        history = collection.get(view)
        history.increment()
        view.add_regions('goto_last_edit_' + str(history.index), view.sel())

class BugCommand(sublime_plugin.TextCommand):

    def run(self, edit, **kargs):
        view = sublime.active_window().active_view();
        BugThread(view).start()

class BugThread(threading.Thread):

    def __init__(self, view):
        self.view = view
        threading.Thread.__init__(self)

    def run(self):
        view = self.view

        for i in range( 0, 1000 ):
            view.run_command("append", {"characters": str( i ) })

        # for i in range( 0, 100 ):
        #     for j in range( 0, 10 ):
        #         view.run_command("append", {"characters": str( j ) })


With @Wramberg pull request the "GotoLastEditEnhanced" history is now limited until certain steps count as 5000:

  1. Unbound history size shagabutdinov/sublime-goto-last-edit-enhanced#3 Unbound history size
  2. Limit size of edit history to a fixed maximum shagabutdinov/sublime-goto-last-edit-enhanced#4 Limit size of edit history to a fixed maximum

However Sublime Text still leaking memory even with that patch. On the following section Sublime Text build 3142, vanilla install, the only package installed on this vanilla install, was the "GotoLastEditEnhanced" patched at (shagabutdinov/sublime-goto-last-edit-enhanced#4):

import sublime
import sublime_plugin

MAX_HIST_SIZE = 5000

class History():
  def __init__(self):
    self.index = 0
    self.start = 0
    self.max = 0

  def remove_oldest(self):
    self.start = self.start + 1
    return (self.start - 1)

  def increment(self):
    self.max += 1
    self.index = self.max

  def size(self):
    return self.max - self.start


class Collection():
  def __init__(self):
    self.list = {}
    self.index = 0

  def get(self, view):
    id = view.id()
    if id not in self.list:
      self.list[id] = History()

    return self.list[id]

collection = Collection()

class GotoLastEditEnhanced(sublime_plugin.TextCommand):
  def run(self, edit, backward = False):
    pass

class Listener(sublime_plugin.EventListener):
  def on_modified(self, view):
    history = collection.get(view)
    if history.size() >= MAX_HIST_SIZE:
      oldest = history.remove_oldest()
      view.erase_regions('goto_last_edit_' + str(oldest))

    history.increment()
    view.add_regions('goto_last_edit_' + str(history.index), view.sel())


Sublime Text build 3142, vanilla install

The performance compared with build 3141, is incredibly higher (the videos above where recorded on build 3141, except this section which is on build 3142). Now Sublime Text stays responsive until it reaches 1.81GB of RAM memory. Then it stops responding, until I close it. Here we may notice the when Sublime Text hanged, it started eating the kernel time (green line) on the CPU graphic:

image

On this animation bellow, it last word typed was Today. After that Sublime Text stayed unresponsive, but no crash reports when asked to be closed:

build3142_hanging


Steps to reproduce

  1. Set up the typer program to type the text I was typing when it originally crashed several times.


  2. After setup the typer program, open a Sublime Text vanilla install and create this package:

    import sublime
    import sublime_plugin
    
    MAX_HIST_SIZE = 5000
    
    class History():
      def __init__(self):
        self.index = 0
        self.start = 0
        self.max = 0
    
      def remove_oldest(self):
        self.start = self.start + 1
        return (self.start - 1)
    
      def increment(self):
        self.max += 1
        self.index = self.max
    
      def size(self):
        return self.max - self.start
    
    
    class Collection():
      def __init__(self):
        self.list = {}
        self.index = 0
    
      def get(self, view):
        id = view.id()
        if id not in self.list:
          self.list[id] = History()
    
        return self.list[id]
    
    collection = Collection()
    
    class GotoLastEditEnhanced(sublime_plugin.TextCommand):
      def run(self, edit, backward = False):
        pass
    
    class Listener(sublime_plugin.EventListener):
      def on_modified(self, view):
        history = collection.get(view)
        if history.size() >= MAX_HIST_SIZE:
          oldest = history.remove_oldest()
          view.erase_regions('goto_last_edit_' + str(oldest))
    
        history.increment()
        view.add_regions('goto_last_edit_' + str(history.index), view.sel())


  3. Now open the Auto Typer to automatically type text on Keyboard.

  4. Configure it as described on the Step 1 to type the text following text:

    Text Used

    
    
    ## Developing your Editor
    
    > @nikhil13 Open Source Sublime Text
    
    This was already posted some years ago, you can read it at:
    
    1. [#7424](https://forum.sublimetext.com/t/sublimes-future-and-open-source/7424) Sublime’s future and open source
    
    
    > @nikhil13 I have been big fan of sublime text-3 and have been using it for 3-4 years
    
    I just completed a year of Sublime Text. On this time I had been working a package toolset to let me use Sublime Text for everything I ever need. You can find it on: https://github.com/evandrocoan/SublimeTextStudio
    
    My goal is to actively develop my text editor while I am working on it. For that I use git to fork all the packages I am using, and soon as I find I bug I can open the source code of the package see what is wrong and create a commit within a fix for it. Therefore I also started the same project for VSCode with similar naming and inspirations: https://github.com/evandrocoan/CodeTextStudio
    
    
    > @nikhil13 But setting up packages, console, input etc can be very exhausting at times.
    
    For example, yesterday I was working while I found a bug on the SideBarEnhancements package, I opened and source code for it, and fixed the problem and pushed a commit to my fork:
    
    1. https://github.com/evandrocoan/SideBarEnhancements/commit/15a1a1785956ee636fd7f0ca09995f3b54f1a0af
    
    What happened? The other day I implemented a feature on my SideBarEnhancements fork, but I did not tested it throughly. So I did not catch a behavior on the Sublime Text core. Hence yesterday after struggling some hours I find out it was a going on the Sublime Text core. Then I filled a new issue on it with the steps to reproduce it:
    
    1. [Core$827](https://github.com/SublimeTextIssues/Core/issues/1827) When the input panel is closed by `enter` the `hide_panel` command is not called
    
    It was labeled as `enhancement`. When I implemented the feature for the SideBarEnhancements I was expecting the command `hide_panel` to be called every time some panel is closed, either by the `enter` key (confirm) or by the `escape` key (cancelled), because the panel is being closed, how else should be panel goes out of screen? But for my surprise the `hide_panel` is currently only called when the panel is closed by the `escape` key, i.e., explicitly called.
    
    Now looking into it, I am not sure it can ever be implemented without breaking backward compatibility. The command `hide_panel` is only issued when it is explicitly called, not when the panel is being "hidden". Now, we are welcome to software development. To be or not to be, that is question.
    
    
    
    ____
    
    ## The Sublime Text and VSCode Development
    
    > @nikhil13 Also updates have been getting really slow lately
    
    Please do not say that. You need to understand, there are only 2 developers working on the core. Not 10 or 100, but 2. The development is going great as it is. I just hope they can focus correctly their valuable and limited time to keep the editor good as it is for the future development.
    
    What is software development? No ninja stuff, unless maybe for Javascript. What is the problem for Sublime Text? Its the C++, you cannot go out there just writing C++ on the speed light. You need to take serious time to understand what you are doing, what you are going to do it, and how are your software tests going to be. Though even Javascript cannot be lend in anyway you would like as pointed out by @braver comment:
    
    > The stuff you really need, e.g. api additions, need so much knowledge, design and coordination that only the contracted developers can do anything about it. You don’t add ligatures or gutter columns on a rainy Sunday afternoon. Are you going to sit down and refactor some thing for a whole week, plus deal with the “+1” comments and update the blog, on a code base you’re not getting paid to maintain? The big pull requests in a project like Atom represent weeks of research, design, discussion and coding.
    
    Why C++ is so hard to work with? For nothing, C++ itself is not entirely the  problem, the problem is the computer you are using and how it works. Why use C++? Because you have the control, explicit control over the computer. It means you can do anything you imagination come with. Which is very good for performance. As long as your are very smart, you can do very smart things and make Sublime Text the fastest thing alive.
    
    However, with great power comes great responsibility. As long you can do very great things, you can do very nasty things and things tend to go very south and you end up with a C++ application slower, or as slower as a Javascript application. Or bad things as frequent/occasional crashes (seg faults). See the issue:
    
    1. [Forum$23042](https://forum.sublimetext.com/t/st3-crashes-frequently-since-build-3124/23042) ST3 crashes frequently since Build 3124
    
    Also doing simple things you could do in Python/Javascript could require much more man-work hours in C++. So, slowing even more the development progress. That is why new higher languages as Python and Javascript, and much more others are invented. To improve the programmer performance, allow them to create more stuff in less time with less bugs.
    
    Also there the proposal of other languages as Rust, D, Ada, etc., trying to be as fast as C++, but less error prone, increasing the programmer code write performance. However I would say his is a field in development today. Now about this field there are several discussions out there. Some stick with one side, some choose stick with the other. In like 20 years from now we may look backward and see what each one of these sides accomplished and gone through. And draw out our conclusions.
    
    
    
    ____
    
    ## The VSCode Development
    
    > I recently discovered VS code
    
    Yo, there is a week I started working with VSCode. Because... It is open source, while Sublime Text is not. Why do I care? Well, because if there is a bug, and there are much bugs on the editor, I cannot fix. But what is my greatest concern are the segmentation faults bugs.
    
    While I was writing this Text on Sublime Text, using my https://github.com/evandrocoan/MarkdownLight syntax fork:
    
    1. Sublime Text crashed 3 times.
    1. I fixed a bug on the MarkdownLight syntax, and created new ones.
    
    The new bugs I created on the MarkdownLight is the not parsing of links on some parts of the text. I will have to look into that again, and see where I should start pushing then on the stack again.
    
    The crash on Sublime Text is on that, I do not know why, why, why......... I did not reported before on the https://github.com/SublimeTextIssues/Core/issues because the dump file had `size 0` so there is nothing to do about it. This was not the first time is crashed like that. I have about 5 or 6 crashes like this, since the fix of the other crash I reported on https://github.com/SublimeTextIssues/Core/issues/1503 (Crash while hovering a File Name on the Tab Bar ).
    
    On that time, the crash always created a dump file I could post, and post, and cry, cry, for the development team to look into it. Now the `dump` file is empty. Hence I have nothing to hold on. But these crashes are more well behaviored than the other, i.e., does not happen too much, often, etc.
    
    When it crashed the second time, while writing this text, I created a new issue on the core within the description for it: https://github.com/SublimeTextIssues/Core/issues/1832 - You can even see the picture of the second love moment:
    
    https://user-images.githubusercontent.com/5332158/28502050-5b7e911c-6fc0-11e7-861d-72c4d872093b.png
    
    Problems like this is what makes the Sublime Text development slower and consume, exhaust, drive away the development time. This would be a issue from Sublime Text I would like to fix. Because I really hate crashes. I definitely would not be using my current Operating System if it was crashing (blue death screen). I need programs and I a system I can trust to never let me down losing my work or just vanishing out some time they fell like wanna do some crashing.
    
    
    
    ____
    
    ## The VSCode Problems
    
    > @nikhil13 although it's UI is not even close to sublime text
    
    I got these problems with their UI:
    
    1\. Comparing Sublime Text gutter with VSCode gutter, we conclude the VSCode gutter is too big. Issue: https://github.com/Microsoft/vscode/issues/30795
    
    <details>
    <p>
    
    **Sublime Text**
    ![image](https://user-images.githubusercontent.com/5332158/28245748-85e48dd6-69e3-11e7-86af-66bd172b158a.png)
    
    **Notepad++**
    ![image](https://user-images.githubusercontent.com/5332158/28245825-685afdd4-69e5-11e7-8213-ce25215f9856.png)
    
    **VSCode**
    ![image](https://user-images.githubusercontent.com/5332158/28245692-17eb3bcc-69e3-11e7-9341-c05cb50aaf06.png)
    
    </p>
    </details><br>
    
    
    
    2\. The font `Consolas 15pt` with `"window.zoomLevel": 1.1,` is rendered narrowed, comparing to Sublime Text with the same font size. Issue: https://github.com/Microsoft/vscode/issues/30794
    
    <details>
    <p>
    
    **VSCode 1.14.1**
    ![image](https://user-images.githubusercontent.com/5332158/28245579-bba795a6-69e0-11e7-87f1-f3728993a6c7.png)
    
    **Sublime Text build 3141**
    ![image](https://user-images.githubusercontent.com/5332158/28245601-19c57162-69e1-11e7-9f66-1db2bec07d49.png)
    
    **Notepad++ 7.4.1**
    ![image](https://user-images.githubusercontent.com/5332158/28245812-fab9e6aa-69e4-11e7-9e82-5fa7d4a525ef.png)
    
    </p>
    </details><br>
    
    
    
    3\. Add border options to `selectionBackground` and `findMatchBackground`. They would be called like `selectionBorder` and `findMatchBorder`. Currently the selection has no border. https://github.com/Microsoft/vscode/issues/30793
    
    <details>
    <p>
    
    ![image](https://user-images.githubusercontent.com/5332158/28245458-8281e9aa-69dd-11e7-8952-ad69839e7cd4.png)
    
    
    Would be awesome if it has borders like Sublime Text selections:
    
    ![image](https://user-images.githubusercontent.com/5332158/28245453-66aa5410-69dd-11e7-912b-868638cb7ffa.png)
    
    </p>
    </details><br>
    
    
    4\. Help tool tip should be placed bellow the mouse cursor on the `settings.json` file, instead of above item. For me it is horribly confusing putting it above it, beyond blocking my vision, as I read what is above my mouse cursor, not bellow it. Perhaps a setting to configure this behavior. https://github.com/Microsoft/vscode/issues/30797
    
    <details>
    <p>
    
    ![image](https://user-images.githubusercontent.com/5332158/28245994-b03f8d9c-69e8-11e7-8208-bb279bd477a2.png)
    
    * The mouse cursor is where the big red arrow is pointing.
    
    </p>
    </details><br>
    
    
    
    5\. > @Gama11 I have the vshaxe extension cloned into .vscode/extensions/vshaxe, which is the usual workflow for working on extensions as far as I can tell. I didn't have the most recent commit pulled yet, so the version in my local package.json was behind the officially relased one. This leads to VSCode actually asking me to update the extension (which doesn't make a lot of sense for an extension installed from source)
        ...
    
    Issue: [vscode$25159](https://github.com/Microsoft/vscode/issues/25159) Support development extension installations
    
    
    ___
    
    ## Big Work Time
    
    > @nikhil13 I resented it earlier
    
    I resented Atom last year when I was looking for Notepad++/Sublime Text alternatives. Atom was too slow. I could not take it. I was very used to Sublime Text and Notepad++ which are incredibly fast. However I got impressed with VSCode. Despite it being also written in Javascript with Electron framework as Atom. It seems faster than Atom, but does not seems as faster as Sublime Text and Notepad++ in general.
    
    For now, I doubt VSCode could be a replacement for Sublime Text because:
    
    > The file will not be displayed in the editor because it is either binary, very large or uses an unsupported text encoding. Issue: [vscode$6474](https://github.com/Microsoft/vscode/issues/6474) VS Code fails to open big files (60MB)
    
    As a Javascript application, still not sure the fact whether is can handle files bigger as 1GB or 5GB, as Sublime Text can handle: [Forum$9832](https://forum.sublimetext.com/t/sublime-text-performance-with-very-large-files/9832/20?u=addons_zz) Sublime Text performance with very large files.
    
    Now I got an Open Source text editor to work with, VSCode. With already several issues to work through. Initially I am going through the:
    
    1. [vscode$25159](https://github.com/Microsoft/vscode/issues/25159) Support development extension installations
    
    Because I just clone the extensions with git on the extensions folder and do all the changes there. Currently when I am contributing to an existent extension, VSCode keep upgrading & overriding my git extension fork with the latest release available on the marketplace. Now I am looking into the VSCode source to find where it is doing such, and cut it out when I have the extension forked as a `git` repository.
    
    > @nikhil13, It would be really nice if you could open source Sublime Text finally, so that all users who are moving to Atom or VS code or bracket due to all these issues could stay.
    
    The problem about Open Sourcing Sublime Text is that it would still a C++ application and segmentation faults are its fidelity. As well said by @facelessuser:
    
    > To me, open source doesn't mean projects get lots of love, it may get more love, but a large percent of users don't want to actually contribute to an open source project except to tell you you to add their beloved feature, or fix this particular issue.
    
    So @nikhil13, what would be your VSCode core features you are planing to get working on? Did you compile the VSCode from its source code or are you just downloading the Microsoft Build? For the first time I downloaded the Microsoft Build, to see how things were. Later when I decided to give a try as an Sublime Text alternative I started looking into how to compile it from the source.
    
    And I would have to say, I got a really hard time trying to compile VSCode from its source code. It just seems anyone else other than the Microsoft development team and some other community gurus does so. I am not a experienced Javascript programmer, I just know the basis. I could not figure out the miracle expected from them to build the source code. The build instructions where not a cook recipe. But after opening a issue on their issue tracker, I found a response from they about what steps should I take. https://github.com/Microsoft/vscode/issues/30909
    
    Comparing the Atom build with the VSCode build, was really different. With Atom I just had to run one command and everything were completed successfully. On the instructions page, they state a lot of stuff. But seems I already had everything ready after the VSCode build, because I only had to run the command:
    ```
    cd C:\
    git clone https://github.com/atom/atom.git
    cd atom
    script\build
    ```
    
    And the final build release of Atom was completed. But for VSCode I still do not know what miracle it is expected to generate a release build. The issue I just pointed there, still with this question in open. But hey, it is open source, I can contribute for the project creating the build steps require to create the release build. But the problem is, how much time will take me to learn it? Why the development team, i.e., the guys who know how to do it, do not did it already? Any ways if some day I got really interested in doing it, I can employ more time. For now, I figured out some other way to deal it the build process, not requiring to build a release version. Therefore I am good and can move on into fixing and creating other things.
    
    
    
    ____
    
    ## The Open Source Initiative
    
    > @facelessuser My personal experience with open source is that you have a small percent of contributors, and a much larger user base that wants a piece of your time for their idea or concern.
    
    Oh, I was that bad boy several times. Though, not everything I reported was something was making trouble to me. That is, when I saw some error popping out, I just take the basis and open a issue on its issue tracker. But just because there is a error, does not mean the error is something breaking my workflow. So, most times I do not stop everything I am doing and start debugging and looking how to fix it.
    
    For example, on the package MarkdownLight I had opened this [issue #13](https://github.com/sekogan/MarkdownLight/issues/13) on Nov 21, 2016. But just today I had it fixed. I did so because I was typing a markdown, and that bug messed with my entire document style. So I had two options:
    
    1. Go on the same issue and post: (Bumping it)
        1. `Please heelllllllpppp`
        1.  `Still not working`
        1. `Any news on this?`
    1. To change the markdown syntax, to some other one.
    1. To use another text editor.
    1. To fix the problem for good.
    
    Today I choose to fix the problem for good. Commit: [evandrocoan/MarkdownLight@2d92bd5](https://github.com/evandrocoan/MarkdownLight/commit/2d92bd50a006768cc23cb6311c67a4fff9f2fba2). But no pull requests made. As there is already one pull request mine, upgrading the syntax from `.tmLanguage` to `.sublime-syntax`.
    
    I usually do not not do pull requests because most developers went MIA (Missing In Action). But sometimes they emerge from the ashes, and got me by surprise. Which by the fact, this is some other feature I need to implement for Sublime Text. Loop through all my forks and check whether the original project owner is starting updating it.
    
    My idea is to do this an automatic process. Except when `git` cannot merge the changes automatically, then he sends me a big warm warning about the unmerged changes. I should do it this vacation, but I decided for now on to move on from Sublime Text and start working with VSCode. I would came back into implement this for Sublime Text on the next vacation time. Because well, now Sublime Text is crashing and I fu***** hate things crashing. So, as VSCode is open an source program, not written in C++, has bigger changes to not let down never.
    
    Why would I think so? Well lets us use the current situation. Sublime Text is crashing sometimes, and there is nothing I can do about, other than cry to the Sublime Text development team. But now let us suppose, VSCode start crashing. Which are my options?
    
    1. Start crying to development team.
    1. Fix the damn thing myself.
    
    Then `Wow, super-boy. The man who fixes everything!` No way, I can barely fix my misspellings errors on this text. However, comparing between the options with VSCode and Sublime Text, I much prefer VSCode because I can much more than stay just sitting and crying waiting for the others to fix my problems.
    
    Also there is more on the equation than just the Sublime Text core crashing. Which are the dedicated time, and efforts I spent on Sublime Text. I have to consider the option that Sublime Text team will consider that my crashing problems are not good enough for their attention because I am the only one complaining about.
    
    Though they are correct, on the business man point of view. Just one user troubling is not enough to spend time looking over his problems. Therefore I also have my own business man point of view. I am, developing several resources and employing huge amounts of time with Sublime Text and my Sublime Text Studio toolset collection. I have to consider that at some point Sublime Text can just became unfixable and I had no other option than not use Sublime Text any more. So, I cannot make Sublime Text, a closed source project, my only escape point.
    
    That is where enters VSCode. I am not saying that VSCode will be my savior and that Sublime Text is dead. Much by the opposite. VSCode could definitely die and my only option would Sublime Text, or something else as `vim`, `Notepad++`, `Atom`, etc. But for now I am going to put on pause the Sublime Text stuff and start diving more into VSCode and develop the Code Text Studio, which is the same toolset I had been preparing for Sublime Text last year. However its has a big difference, it includes the source VSCode on the installation step. Which does not happen on the Sublime Text version, as it is a closed source project.
    
    I hope this increases my changes of success, however we also need to increase the point of view. I am not the only one the equation, but just a very tiny small portion of the game. The ones who really matter are the big mass of developers. They decide how the game ends, who wins and are the losers.
    
    Whether VSCode is going to be a success depends on the the masses adheres to its use and engages on its development. Also counts how much more time Microsoft pretends to employ its man power on develop the VSCode for free. If Microsoft stopped working on it, I am not sure about how much time would last the development of VSCode. Maybe the community employment (me, you, everybody else who works for free) should be enough. May be not. Issue: https://github.com/Microsoft/vscode/issues/31289 (How much time Microsoft pretends to employ on VSCode?).
    
    
    
    


  5. Open some program to monitor the Sublime Text CPU and RAM usage.

  6. Focus on an empty Sublime Text view and press F8 to the typer program start typing.


Environment

  • Operating system and version:
    • Windows 10 build 15063 x64
    • Mac OS ...
    • Linux ...
  • Monitor:
    • Resolution 1920x1080
    • dpi_scale used in ST 1.0

The videos presented here, where recorded on Sublime Text build 3141. But on the middle of the time I as writing this issue, the Sublime Text build 3142 was released. Then I created a new topic called Sublime Text build 3142, vanilla install testing this issue on the new build 3142.

  • Sublime Text:
    • Build 3141
    • 32 bit

Related issues:

  1. Several crashes while typing text #1832 Several crashes while typing text
  2. ST slowing down while continously appending text (to a single line) #1845 Python package slowing down while calling run_command("append")
  3. Unbound history size shagabutdinov/sublime-goto-last-edit-enhanced#3 Unbound history size
@keith-hall
Copy link
Collaborator

One of the ST developers briefly mentioned that ST expects that there won't be many unique add_region keys in a single view here so marking as an enhancement.

@deathaxe
Copy link
Collaborator

Possibly related with the out of memory exception described here: https://forum.sublimetext.com/t/security-flaw-cve-2017-8368/30403/5

@BenjaminSchaaf
Copy link
Member

This no longer reproduces in ST4:

Screencast.from.2023-10-26.15-20-32.webm

@evandrocoan
Copy link
Author

evandrocoan commented Oct 26, 2023

Your RAM jumped from 25.5GB to 26.4GB (900MB). In my original post, I am only monitoring the RAM usage of Sublime Text in my system. So, your reproduction should make Sublime Text use more than 1.81GB instead of 0.9GB. I was also using Sublime Text x86 for Windows instead of Sublime x64. x86 application usually cannot use more RAM because of the limitations of x86 architecture.

The limitation of x86 RAM usage is not a problem, just that add_regions uses too much RAM.

@BenjaminSchaaf
Copy link
Member

Unsurprisingly adding N regions for M undo states results in N * M memory usage. This isn't fixable on our end, the amount of data the plugin is asking us to store is the problem. This can be avoided by passing the sublime.RegionFlags.NO_UNDO flag to add_regions.

@evandrocoan
Copy link
Author

Thanks for looking into it. Next time I will try using this RegionFlags.NO_UNDO to reduce memory usage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants