Ticket #3 (new enhancement)

Opened 8 years ago

Last modified 16 months ago

savannah: VFS optimization request

Reported by: slavazanko Owned by:
Priority: minor Milestone: Future Releases
Component: mc-vfs Version: master
Keywords: Cc:
Blocked By: Blocking: #1952
Branch state: no branch Votes for changeset:

Description (last modified by ossi) (diff)

Original: http://savannah.gnu.org/bugs/?8136

Submitted by:NoneSubmitted on:Fri 12 Mar 2004 11:00:45 PM UTC
Category:VFSSeverity:3 - Normal
Status:NonePrivacy:Public
Assigned to:NoneOpen/Closed:Open
Release:4.6.0Operating System:GNU/Linux

Original submission:

Not a bug, but feature request.

I am using VFS (extfs). I have written program (backup extractor) 
which handle "copyout" and "list" commands from mc. But if I want to
 copyout a whole directory (or a lot of files from archive), it is 
called FOR EACH extracted file. In my case, extractor (duplicity 
backupper, http://www.nongnu.org/duplicity/ ) starts very long time.
 But it can extract whole directory quickly.

I think if mc would pass to extfs script environment variable 
containing names of ALL the selected files (AND DIRECTORIES) on the 
panel before extracting, it would get ability to optimize extract 
script. For example, I can fully extract all needed files in 
temporary directory by ONE call to duplicity, and then just copy 
these files from temp directory to pathes needed by mc.

Comment 1 by Anonymous at Sat 13 Mar 2004 12:19:29 AM UTC:

Here is a simple patch (also attached). Hope you will apply it in
next release.

--- mc-4.6.1-pre1/src/file.c 2003-10-27 20:36:19.000000000 +0300
+++ mc-4.6.1-pre1.patched/src/file.c 2004-03-12 03:13:05.000000000 +0300
@@ -1865,4 +1865,27 @@
}
}
+
+ /*
+ * Dmitry Koteroff <d[at]koteroff.ru>
+ * Patch to save selected items in environment variable.
+ * This variable would be passed to extfs scripts and may be used
+ * to optimize depacking performance.
+ */
+ {{{{
+ char* env = g_strconcat("", NULL);
+ for (i = 0; i < panel->count; i++) {
+ char prevEnv, p;
+ if (!panel->dir.list[i].f.marked) continue;
+ p = concat_dir_and_file (panel->cwd, panel->dir.list[i].fname);
+ prevEnv = env;
+ env = g_strconcat(prevEnv, p, "\n", NULL);
+ g_free(p);
+ g_free(prevEnv);
+ }
+ setenv("MC_SELECTION", env, 1);
+ g_free(env);
+ }}}}
+
+
#ifdef WITH_BACKGROUND
/* Did the user select to do a background operation? */

Comment 2 by Anonymous at Sat 13 Mar 2004 06:40:31 PM UTC:

Here is the better patch, it also consider current cursor position.

P.S.
"return 0" in background operation instead of "goto glean_up" isn't a bug?..

--- mc-4.6.0/src/file.c 2002-12-26 22:04:10.000000000 +0300
+++ mc-4.6.0.patched/src/file.c 2004-03-12 21:22:17.000000000 +0300
@@ -1895,4 +1895,30 @@
}
}
+
+ /*
+ * Dmitry Koteroff <d[at]koteroff.ru>
+ * Patch to save selected items in environment variable.
+ * This variable would be passed to extfs scripts and may be used
+ * to optimize depacking performance.
+ */
+ {{{{
+ char* env = g_strconcat("", NULL);
+ for (i = 0; i < panel->count; i++) {
+ char prevEnv, p;
+ if (!panel->dir.list[i].f.marked) continue;
+ p = concat_dir_and_file (panel->cwd, panel->dir.list[i].fname);
+ prevEnv = env;
+ env = g_strconcat(prevEnv, p, "\n", NULL);
+ g_free(p);
+ g_free(prevEnv);
+ }
+ setenv("MC_SELECTION", env, 1);
+ g_free(env);
+ char *cur = concat_dir_and_file (panel->cwd, source);
+ setenv("MC_CURRENT", cur, 1);
+ g_free(cur);
+ }}}}
+
+
#ifdef WITH_BACKGROUND
/* Did the user select to do a background operation? */
@@ -1913,4 +1939,5 @@
mc_setctl (dest, MCCTL_FORGET_ABOUT, NULL);
/* file_op_context_destroy (ctx); */
+ setenv("MC_SELECTION", "", 1); setenv("MC_CURRENT", "", 1);
return 0;
}
@@ -2123,4 +2150,5 @@
} /* Many files */
clean_up:
+ setenv("MC_SELECTION", "", 1); setenv("MC_CURRENT", "", 1);
/* Clean up */

Attachments

mc-4.6.0.diff (1.3 KB) - added by slavazanko 8 years ago.
mc-4.6.1-pre1.diff.zip (694 bytes) - added by slavazanko 8 years ago.

Change History

Changed 8 years ago by slavazanko

Changed 8 years ago by slavazanko

comment:1 Changed 8 years ago by metux

Wouldnt it be better to introduce some new vfs operation
(bulk_copy?) for that ?

comment:2 Changed 8 years ago by styx

  • Milestone set to 4.7

comment:3 Changed 7 years ago by styx

  • Milestone changed from 4.7 to VFS Standardisation

comment:4 Changed 6 years ago by birdie

  • severity set to no branch

Also I have a firm belief that VFS plugins must be rewritten using a faster (than shell) language. It's already done for CPIO and TAR but other archivers need this treatment too. However I have no idea which language can suit this task because C/C++ language is difficult to code and patch quickly, while script languages are not really as fast as they need to be.

Try to enter an archive containing over 50 thousand files and you'll instantly notice how much time MC spends parsing a VFS plugin output.

comment:5 Changed 6 years ago by slavazanko

  • Blocking 1952 added

(In #1952) Ticket frozen until ticket:3 unfixed.

comment:6 Changed 5 years ago by TiGR

  • Branch state set to no branch

Are there any news on this ticket? Currently, unpacking archives is a real pain, while it is a very common use case.

comment:7 Changed 5 years ago by andrew_b

A lot of changes in VFS layer are required for this issue. That will be possible only after 4.8.0 release.

comment:8 Changed 3 years ago by ossi

  • Description modified (diff)

comment:9 Changed 16 months ago by andrew_b

  • Version set to master
  • Milestone changed from 4.8 to Future Releases
Note: See TracTickets for help on using tickets.