Michel Lespinasse 38a76013ad
mm: avoid taking rmap locks in move_ptes() ...
During mremap(), the destination VMA is generally placed after the
original vma in rmap traversal order: in move_vma(), we always have
new_pgoff >= vma->vm_pgoff, and as a result new_vma->vm_pgoff >=
vma->vm_pgoff unless vma_merge() merged the new vma with an adjacent one.
When the destination VMA is placed after the original in rmap traversal
order, we can avoid taking the rmap locks in move_ptes().
Essentially, this reintroduces the optimization that had been disabled in
"mm anon rmap: remove anon_vma_moveto_tail". The difference is that we
don't try to impose the rmap traversal order; instead we just rely on
things being in the desired order in the common case and fall back to
taking locks in the uncommon case. Also we skip the i_mmap_mutex in
addition to the anon_vma lock: in both cases, the vmas are traversed in
increasing vm_pgoff order with ties resolved in tree insertion order.
Signed-off-by: Michel Lespinasse <walken@google.com >
Cc: Andrea Arcangeli <aarcange@redhat.com >
Cc: Rik van Riel <riel@redhat.com >
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl >
Cc: Daniel Santos <daniel.santos@pobox.com >
Cc: Hugh Dickins <hughd@google.com >
Signed-off-by: Andrew Morton <akpm@linux-foundation.org >
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org >
2012-10-09 16:22:42 +09:00
2012-10-02 17:26:42 -07:00
2012-10-03 13:45:43 -07:00
2012-07-20 12:31:05 +02:00
2012-10-08 06:38:18 +09:00
2012-09-19 19:08:53 -06:00
2012-10-02 18:01:25 +01:00
2012-09-27 14:28:09 -03:00
2012-09-05 12:20:49 -07:00
2012-10-09 07:07:14 +09:00
2012-09-17 21:48:51 +01:00
2012-10-02 17:16:10 -07:00
2012-09-19 19:08:37 -06:00
2012-07-27 16:49:22 -04:00
2012-10-09 07:07:14 +09:00
2012-09-30 20:33:43 -07:00
2012-09-04 13:58:10 -04:00
2012-09-04 09:38:58 +03:00
2012-10-03 13:45:43 -07:00
2012-08-30 03:00:25 +02:00
2012-07-31 01:02:51 +04:00
2012-10-03 13:45:43 -07:00
2012-10-09 07:07:14 +09:00
2012-10-07 17:29:24 +09:00
2012-09-28 14:45:07 +01:00
2012-08-17 22:54:15 +01:00
2012-08-10 15:27:02 -04:00
2012-09-07 11:43:49 -04:00
2012-09-18 17:40:56 +01:00
2012-09-24 16:36:10 -06:00
2012-07-30 17:25:21 -07:00
2012-07-20 09:07:37 +01:00
2012-09-13 01:08:53 -04:00
2012-09-11 08:56:37 +02:00
2012-08-28 10:14:01 -07:00
2012-10-06 03:04:35 +09:00
2012-08-04 01:24:44 +04:00
2012-09-10 11:13:16 -07:00
2012-09-19 19:08:46 -06:00
2012-10-06 03:05:16 +09:00
2012-07-31 18:42:47 -07:00
2012-08-09 15:20:23 +02:00
2012-08-01 12:24:18 +02:00
2012-08-07 10:10:57 +02:00
2012-07-17 21:37:27 +02:00
2012-09-14 09:57:37 -07:00
2012-10-02 10:52:28 -07:00
2012-09-07 18:30:19 -07:00
2012-07-30 17:25:11 -07:00
2012-09-04 01:36:01 +02:00
2012-10-09 16:22:21 +09:00
2012-10-06 03:05:16 +09:00
2012-09-17 15:00:38 -07:00
2012-09-17 15:00:38 -07:00
2012-10-06 03:05:16 +09:00
2012-07-19 08:34:37 -07:00
2012-07-17 12:39:26 -07:00
2012-08-23 17:16:42 +02:00
2012-07-27 09:29:42 +09:30
2012-07-24 13:53:22 +02:00
2012-09-18 11:23:51 -07:00
2012-07-27 15:08:16 +01:00
2012-10-02 18:32:35 -07:00
2012-07-27 15:08:16 +01:00
2012-07-30 12:25:47 +02:00
2012-07-30 12:25:46 +02:00
2012-09-24 18:35:29 +02:00
2012-10-02 18:01:25 +01:00
2012-07-19 10:39:00 +02:00
2012-09-29 12:21:03 -07:00
2012-09-29 12:21:03 -07:00
2012-10-06 03:05:17 +09:00
2012-09-10 15:49:54 -04:00
2012-08-21 01:19:54 -07:00
2012-10-06 03:05:00 +09:00
2012-09-05 12:18:11 -07:00
2012-09-27 09:29:33 -04:00
2012-09-26 21:10:00 -04:00
2012-09-26 22:20:08 -04:00
2012-09-24 16:49:21 -04:00
2012-09-28 11:47:42 +02:00
2012-08-16 13:24:21 -07:00
2012-09-21 10:38:12 -04:00
2012-10-09 16:22:39 +09:00
2012-09-24 13:42:45 -07:00
2012-07-31 17:02:05 +02:00
2012-07-31 10:29:57 -04:00
2012-07-18 16:09:40 +02:00
2012-10-06 03:04:57 +09:00
2012-07-24 00:01:30 -07:00
2012-08-01 09:02:41 -07:00
2012-10-09 16:22:15 +09:00
2012-09-25 15:31:31 +02:00
2012-08-09 16:18:06 -07:00
2012-09-06 19:13:13 +01:00
2012-09-06 19:13:13 +01:00
2012-09-19 19:50:20 +02:00
2012-07-31 18:42:47 -07:00
2012-10-09 16:22:29 +09:00
2012-07-31 18:42:41 -07:00
2012-07-31 18:42:50 -07:00
2012-09-10 16:42:33 -07:00
2012-10-05 22:23:54 +02:00
2012-10-05 22:23:51 +02:00
2012-09-12 17:52:44 +02:00
2012-10-05 22:23:54 +02:00
2012-10-06 03:04:56 +09:00
2012-09-06 17:05:36 +02:00
2012-08-14 14:28:32 -07:00
2012-10-01 18:39:45 -04:00
2012-08-22 14:21:38 -07:00
2012-08-14 14:28:32 -07:00
2012-08-24 13:46:39 -04:00
2012-09-07 14:57:47 -04:00
2012-08-14 21:55:20 -07:00
2012-08-04 01:27:57 -07:00
2012-09-17 18:08:54 -07:00
2012-10-06 03:04:37 +09:00
2012-09-19 19:50:18 +02:00
2012-09-07 14:57:45 -04:00
2012-09-23 07:42:52 -07:00
2012-10-09 16:22:40 +09:00
2012-10-09 16:22:39 +09:00
2012-09-25 12:14:56 +02:00
2012-09-11 10:15:02 +02:00
2012-08-14 14:28:32 -07:00
2012-09-06 22:17:20 -07:00
2012-08-30 03:00:10 +02:00
2012-07-25 12:46:38 +02:00
2012-10-01 09:06:36 -07:00
2012-08-05 19:04:57 -04:00
2012-08-22 14:21:38 -07:00
2012-07-17 09:13:37 -07:00
2012-10-07 17:49:05 +09:00
2012-09-26 13:42:26 -07:00
2012-07-30 17:25:14 -07:00
2012-09-25 15:42:37 +02:00
2012-09-19 06:51:25 -07:00
2012-07-30 15:08:46 +10:00
2012-09-13 18:28:02 -07:00
2012-09-26 13:52:36 -07:00
2012-07-21 10:34:00 -07:00
2012-09-06 10:13:58 -07:00
2012-07-31 10:29:59 -04:00
2012-08-22 10:24:41 -04:00
2012-08-13 17:01:06 +02:00
2012-08-15 15:54:01 +02:00
2012-10-04 09:30:33 -07:00
2012-09-23 13:50:15 +02:00
2012-08-13 16:45:48 +02:00
2012-10-02 19:01:32 -07:00
2012-07-23 13:54:52 +01:00
2012-10-06 02:48:09 +09:00
2012-09-21 03:13:20 -07:00
2012-08-22 22:58:27 -07:00
2012-07-31 18:42:48 -07:00
2012-09-17 15:00:37 -07:00
2012-10-09 16:22:22 +09:00
2012-09-24 15:54:33 -04:00
2012-07-31 18:42:40 -07:00
2012-07-19 15:51:36 -04:00
2012-09-13 14:58:54 -04:00
2012-10-09 16:22:39 +09:00
2012-10-09 16:22:42 +09:00
2012-10-09 16:22:18 +09:00
2012-10-09 16:22:23 +09:00
2012-07-31 18:42:46 -07:00
2012-10-01 18:02:07 -07:00
2012-08-15 13:58:09 +00:00
2012-07-29 21:24:13 +04:00
2012-10-06 03:05:24 +09:00
2012-10-02 20:25:04 -07:00
2012-10-02 13:38:27 -07:00
2012-08-30 03:00:14 +02:00
2012-08-30 03:00:14 +02:00
2012-09-23 02:09:23 -04:00
2012-08-14 14:33:32 -07:00
2012-09-27 10:47:59 +02:00
2012-07-30 19:06:52 -04:00
2012-09-04 14:52:42 -04:00
2012-07-30 19:06:52 -04:00
2012-08-02 17:41:18 -04:00
2012-09-06 11:11:53 -04:00
2012-07-30 17:25:19 -07:00
2012-09-18 19:54:06 +02:00
2012-07-27 13:57:23 -04:00
2012-08-01 17:47:56 +08:00
2012-08-03 08:01:46 -05:00
2012-09-20 12:32:30 +02:00
2012-08-30 12:37:52 -04:00
2012-10-05 12:01:30 +09:00
2012-07-31 23:27:30 -04:00
2012-08-11 15:59:54 -03:00
2012-07-31 12:06:20 +01:00
2012-09-10 16:34:00 -07:00
2012-10-09 16:22:24 +09:00
2012-09-09 22:06:33 +02:00
2012-08-20 02:23:14 -07:00
2012-07-31 18:42:43 -07:00
2012-07-31 18:42:45 -07:00
2012-07-31 18:42:45 -07:00
2012-07-31 18:42:47 -07:00
2012-10-05 22:23:53 +02:00
2012-08-23 18:32:36 -06:00
2012-09-24 14:51:23 -06:00
2012-09-13 09:08:02 -06:00
2012-09-07 16:24:59 -06:00
2012-10-06 03:04:44 +09:00
2012-10-01 10:28:49 -07:00
2012-08-10 11:31:26 -03:00
2012-08-01 10:26:23 -07:00
2012-08-16 10:32:07 -07:00
2012-09-04 01:36:53 +02:00
2012-07-19 00:02:30 +02:00
2012-10-02 18:32:35 -07:00
2012-09-18 01:01:35 -07:00
2012-09-18 01:01:35 -07:00
2012-07-26 13:36:43 -07:00
2012-08-22 21:59:15 -07:00
2012-09-07 21:13:28 +01:00
2012-07-30 17:25:14 -07:00
2012-09-18 01:01:37 -07:00
2012-07-17 16:48:09 -07:00
2012-09-06 22:16:58 -07:00
2012-09-22 15:42:46 -04:00
2012-08-03 14:47:17 +10:00
2012-07-23 13:23:52 +02:00
2012-09-18 01:01:41 -07:00
2012-09-18 01:01:40 -07:00
2012-07-19 10:38:32 -04:00
2012-10-09 16:22:40 +09:00
2012-10-09 16:22:40 +09:00
2012-09-26 15:47:02 +02:00
2012-08-30 11:13:41 -07:00
2012-09-18 21:55:52 +03:00
2012-08-21 20:50:25 +02:00
2012-10-06 03:05:21 +09:00
2012-10-06 03:05:23 +09:00
2012-10-09 16:22:42 +09:00
2012-10-06 03:05:01 +09:00
2012-10-06 03:05:04 +09:00
2012-07-30 12:25:44 +02:00
2012-10-09 16:22:24 +09:00
2012-09-17 13:29:23 +01:00
2012-10-02 21:38:48 -07:00
2012-08-14 21:47:55 -07:00
2012-07-17 09:11:50 -07:00
2012-09-26 13:47:02 -07:00
2012-09-06 09:17:01 -07:00
2012-09-05 13:15:07 -07:00
2012-07-20 11:28:20 +05:30
2012-08-01 13:48:52 +09:00
2012-07-30 17:25:20 -07:00
2012-08-24 15:55:33 -07:00
2012-07-31 18:42:43 -07:00
2012-08-04 01:27:57 -07:00
2012-09-29 10:02:27 +03:00
2012-09-25 10:14:18 +03:00
2012-09-25 10:11:14 +03:00
2012-08-13 17:01:07 +02:00
2012-08-31 20:02:18 -04:00
2012-07-16 22:31:34 -07:00
2012-07-19 11:02:03 -07:00
2012-09-22 16:35:13 +03:00
2012-08-31 16:11:28 -04:00
2012-08-21 16:45:03 -07:00
2012-07-31 18:42:47 -07:00
2012-08-21 14:47:25 -04:00
2012-09-13 16:47:34 +02:00
2012-09-05 15:15:02 -04:00
2012-09-20 17:12:27 -04:00
2012-07-31 18:08:25 -07:00
2012-08-16 10:11:45 -07:00
2012-09-01 10:24:48 -07:00
2012-08-21 16:28:31 +02:00
2012-10-09 16:22:32 +09:00
2012-07-31 17:09:12 +02:00
2012-08-20 02:26:31 -07:00
2012-08-13 19:02:05 +02:00
2012-08-22 11:11:33 -05:00
2012-07-22 23:57:55 +04:00
2012-09-12 10:20:14 -04:00
2012-09-18 01:01:32 -07:00
2012-08-13 16:45:39 -07:00
2012-09-05 13:11:03 -07:00
2012-10-02 11:11:09 -07:00
2012-07-22 23:57:56 +04:00
2012-08-21 22:29:54 -07:00
2012-09-15 17:37:28 +02:00
2012-09-05 17:21:36 -07:00
2012-09-10 13:04:01 -07:00
2012-09-10 11:10:37 -07:00
2012-09-18 01:01:37 -07:00
2012-07-31 00:32:28 -03:00
2012-09-26 09:25:41 -03:00
2012-09-13 16:05:27 -03:00
2012-07-31 08:16:24 -06:00
2012-09-26 09:49:37 -03:00
2012-07-30 13:30:52 +09:30
2012-09-28 15:05:15 +09:30
2012-07-30 13:30:49 +09:30
2012-09-28 15:05:15 +09:30
2012-07-20 08:58:58 +01:00
2012-09-28 15:05:15 +09:30
2012-07-31 18:42:46 -07:00
2012-07-30 12:25:45 +02:00
2012-07-31 18:42:50 -07:00
2012-08-16 10:02:33 -07:00
2012-08-21 13:18:24 -07:00
2012-08-04 12:15:37 +04:00
2012-10-02 21:38:48 -07:00
2012-09-20 18:08:40 -04:00
38a76013ad