栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 面试经验 > 面试问答

如何有效(性能)从Java中的列表中删除许多项?

面试问答 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

如何有效(性能)从Java中的列表中删除许多项?

好的,现在该是提出的方法的测试结果了。这里是我测试过的方法(每种方法的名称在我的资源中也是类名):

  1. NaiveRemoveManyPerformer
    -
    ArrayList
    使用迭代器和删除-我的问题中给出的第一个天真的实现。
  2. BetterNaiveRemoveManyPerformer
    -
    ArrayList
    向后迭代并从头到尾删除。
  3. linkedRemoveManyPerformer
    -天真的迭代器,并删除但仍在进行
    linkedList
    。障碍:仅适用于
    linkedList
  4. CreateNewRemoveManyPerformer
    -
    ArrayList
    作为副本制作(仅添加保留的元素),迭代器用于遍历input
    ArrayList
  5. SmartCreateNewRemoveManyPerformer
    -更好
    CreateNewRemoveManyPerformer
    -结果的初始大小(容量)
    ArrayList
    设置为最终列表大小。缺点:启动时必须知道列表的最终大小。
  6. FasterSmartCreateNewRemoveManyPerformer
    -更好(?)
    SmartCreateNewRemoveManyPerformer
    -使用项目索引(
    items.get(idx)
    )代替迭代器。
  7. MagicRemoveManyPerformer
    -
    ArrayList
    从列表末尾的项目开始就位(无列表副本)并压缩孔(已删除的项目)。缺点:此方法更改列表中项目的顺序。
  8. ForwardInPlaceRemoveManyPerformer
    -适用于
    ArrayList
    -移动保留项以填充孔,最后返回subList(没有最终移除或清除)。
  9. GuavaArrayListRemoveManyPerformer
    -Google Guava
    Iterables.removeIf
    用于
    ArrayList
    -几乎与
    ForwardInPlaceRemoveManyPerformer
    列表末尾相同,但最终将其删除。

完整的源代码在此答案的末尾给出。

使用不同的列表大小(从10,000项到10,000,000项)和不同的删除因子(指定必须从列表中删除多少个项目)执行测试。

正如我在此处的评论中发布的其他答案一样,我认为将项目从复制

ArrayList
到秒
ArrayList
将比迭代
linkedList
和删除项目要快。Sun的Java文档说,
ArrayList
linkedList
实现相比,常数因子低,但是令人惊讶的是,我的问题并非如此。

实际上

linkedList
,在大多数情况下,简单的迭代和删除具有最佳性能(此方法在中实现
linkedRemoveManyPerformer
)。通常,只有
MagicRemoveManyPerformer
性能可与媲美
linkedRemoveManyPerformer
,而其他方法则要慢得多。Google
Guava
GuavaArrayListRemoveManyPerformer
比手工制作的类似代码慢(因为我的代码不会删除列表末尾的不必要项目)。

从1,000,000个源项目中删除500,000个项目的示例结果:

  1. NaiveRemoveManyPerformer
    :未执行测试-我不是那个病人,但是它的表现比差
    BetterNaiveRemoveManyPerformer
  2. BetterNaiveRemoveManyPerformer
    :226080毫
  3. linkedRemoveManyPerformer
    :69毫
  4. CreateNewRemoveManyPerformer
    :246毫
  5. SmartCreateNewRemoveManyPerformer
    :112毫
  6. FasterSmartCreateNewRemoveManyPerformer
    :202毫
  7. MagicRemoveManyPerformer
    :74毫
  8. ForwardInPlaceRemoveManyPerformer
    :69毫
  9. GuavaArrayListRemoveManyPerformer
    :118毫

从1,000,000个源项目中删除1个项目的示例结果(删除第一个项目):

  1. BetterNaiveRemoveManyPerformer:34毫
  2. linkedRemoveManyPerformer:41毫
  3. CreateNewRemoveManyPerformer:253毫
  4. SmartCreateNewRemoveManyPerformer:108毫
  5. FasterSmartCreateNewRemoveManyPerformer:71毫
  6. MagicRemoveManyPerformer:43毫
  7. ForwardInPlaceRemoveManyPerformer:73毫
  8. GuavaArrayListRemoveManyPerformer:78毫

从1,000,000个源项目中删除333,334个项目的示例结果:

  1. BetterNaiveRemoveManyPerformer:253206毫
  2. linkedRemoveManyPerformer:69毫
  3. CreateNewRemoveManyPerformer:245毫
  4. SmartCreateNewRemoveManyPerformer:111毫
  5. FasterSmartCreateNewRemoveManyPerformer:203毫
  6. MagicRemoveManyPerformer:69毫
  7. ForwardInPlaceRemoveManyPerformer:72毫升
  8. GuavaArrayListRemoveManyPerformer:102毫

从1,000,000个源项目中删除1,000,000个(所有)项目的示例结果(所有项目都被删除,但经过一个接一个的处理,如果您先验地知道要删除所有项目,则应该简单地清除列表):

  1. BetterNaiveRemoveManyPerformer:58毫
  2. linkedRemoveManyPerformer:88毫
  3. CreateNewRemoveManyPerformer:95毫
  4. SmartCreateNewRemoveManyPerformer:91毫
  5. FasterSmartCreateNewRemoveManyPerformer:48毫
  6. MagicRemoveManyPerformer:61毫
  7. ForwardInPlaceRemoveManyPerformer:49毫
  8. GuavaArrayListRemoveManyPerformer:133毫

我的最终结论是:使用混合方法-如果处理linkedList-最好进行简单的迭代和删除,如果处理ArrayList-取决于项目顺序是否重要-
然后使用ForwardInPlaceRemoveManyPerformer-如果项目顺序可能更改-
最佳选择是MagicRemoveManyPerformer。如果先验地知道移除因子(您知道要移除多少个项目还是保留多少个项目),那么在某些情况下可以采用更多条件选择效果更好的方法。但是已知的去除因子不是通常的情况…
Google Guava

Iterables.removeIf
是一种混合解决方案,但是假设稍有不同(必须更改原始列表,无法创建新列表,并且始终按物料顺序排列)-这些是最常见的假设,因此
removeIf
最好大多数现实生活中的选择。

还要注意,所有好的方法(天真的不好!)都足够好-在实际应用中,其中任何一个都做得很好,但是必须避免使用天真的方法。

最后-我的测试源代码。

package WildWezyrListRemovalTesting;import com.google.common.base.Predicate;import com.google.common.collect.Iterables;import java.util.ArrayList;import java.util.Iterator;import java.util.linkedList;import java.util.List;public class RemoveManyFromList {    public static abstract class baseRemoveManyPerformer {        protected String performerName() { return getClass().getSimpleName();        }        protected void info(String msg) { System.out.println(performerName() + ": " + msg);        }        protected void populateList(List<Integer> items, int itemCnt) { for (int i = 0; i < itemCnt; i++) {     items.add(i); }        }        protected boolean mustRemoveItem(Integer itemVal, int itemIdx, int removeFactor) { if (removeFactor == 0) {     return false; } return itemIdx % removeFactor == 0;        }        protected abstract List<Integer> removeItems(List<Integer> items, int removeFactor);        protected abstract List<Integer> createInitialList();        public void testMe(int itemCnt, int removeFactor) { List<Integer> items = createInitialList(); populateList(items, itemCnt); long startMillis = System.currentTimeMillis(); items = removeItems(items, removeFactor); long endMillis = System.currentTimeMillis(); int chksum = 0; for (Integer item : items) {     chksum += item; } info("removing took " + (endMillis - startMillis)         + " milli(s), itemCnt=" + itemCnt         + ", removed items: " + (itemCnt - items.size())         + ", remaining items: " + items.size()         + ", checksum: " + chksum);        }    }    private List<baseRemoveManyPerformer> rmps = new ArrayList<baseRemoveManyPerformer>();    public void addPerformer(baseRemoveManyPerformer rmp) {        rmps.add(rmp);    }    private Runtime runtime = Runtime.getRuntime();    private void runGc() {        for (int i = 0; i < 5; i++) { runtime.gc();        }    }    public void testAll(int itemCnt, int removeFactor) {        runGc();        for (baseRemoveManyPerformer rmp : rmps) { rmp.testMe(itemCnt, removeFactor);        }        runGc();        System.out.println("n--------------------------n");    }    public static class NaiveRemoveManyPerformer extends baseRemoveManyPerformer {        @Override        public List<Integer> removeItems(List<Integer> items, int removeFactor) { if (items.size() > 300000 && items instanceof ArrayList) {     info("this removeItems is too slow, returning without processing");     return items; } int i = 0; Iterator<Integer> iter = items.iterator(); while (iter.hasNext()) {     Integer item = iter.next();     if (mustRemoveItem(item, i, removeFactor)) {         iter.remove();     }     i++; } return items;        }        @Override        public List<Integer> createInitialList() { return new ArrayList<Integer>();        }    }    public static class BetterNaiveRemoveManyPerformer extends NaiveRemoveManyPerformer {        @Override        public List<Integer> removeItems(List<Integer> items, int removeFactor) {// if (items.size() > 300000 && items instanceof ArrayList) {//     info("this removeItems is too slow, returning without processing");//     return items;// } for (int i = items.size(); --i >= 0;) {     Integer item = items.get(i);     if (mustRemoveItem(item, i, removeFactor)) {         items.remove(i);     } } return items;        }    }    public static class linkedRemoveManyPerformer extends NaiveRemoveManyPerformer {        @Override        public List<Integer> createInitialList() { return new linkedList<Integer>();        }    }    public static class CreateNewRemoveManyPerformer extends NaiveRemoveManyPerformer {        @Override        public List<Integer> removeItems(List<Integer> items, int removeFactor) { List<Integer> res = createResultList(items, removeFactor); int i = 0; for (Integer item : items) {     if (mustRemoveItem(item, i, removeFactor)) {         // no-op     } else {         res.add(item);     }     i++; } return res;        }        protected List<Integer> createResultList(List<Integer> items, int removeFactor) { return new ArrayList<Integer>();        }    }    public static class SmartCreateNewRemoveManyPerformer extends CreateNewRemoveManyPerformer {        @Override        protected List<Integer> createResultList(List<Integer> items, int removeFactor) { int newCapacity = removeFactor == 0 ? items.size()         : (int) (items.size() * (removeFactor - 1L) / removeFactor + 1); //System.out.println("newCapacity=" + newCapacity); return new ArrayList<Integer>(newCapacity);        }    }    public static class FasterSmartCreateNewRemoveManyPerformer extends SmartCreateNewRemoveManyPerformer {        @Override        public List<Integer> removeItems(List<Integer> items, int removeFactor) { List<Integer> res = createResultList(items, removeFactor); for (int i = 0; i < items.size(); i++) {     Integer item = items.get(i);     if (mustRemoveItem(item, i, removeFactor)) {         // no-op     } else {         res.add(item);     } } return res;        }    }    public static class ForwardInPlaceRemoveManyPerformer extends NaiveRemoveManyPerformer {        @Override        public List<Integer> removeItems(List<Integer> items, int removeFactor) { int j = 0; // destination idx for (int i = 0; i < items.size(); i++) {     Integer item = items.get(i);     if (mustRemoveItem(item, i, removeFactor)) {         // no-op     } else {         if (j < i) {  items.set(j, item);         }         j++;     } } return items.subList(0, j);        }    }    public static class MagicRemoveManyPerformer extends NaiveRemoveManyPerformer {        @Override        public List<Integer> removeItems(List<Integer> items, int removeFactor) { for (int i = 0; i < items.size(); i++) {     if (mustRemoveItem(items.get(i), i, removeFactor)) {         Integer retainedItem = removeSomeFromEnd(items, removeFactor, i);         if (retainedItem == null) {  items.remove(i);  break;         }         items.set(i, retainedItem);     } } return items;        }        private Integer removeSomeFromEnd(List<Integer> items, int removeFactor, int lowerBound) { for (int i = items.size(); --i > lowerBound;) {     Integer item = items.get(i);     items.remove(i);     if (!mustRemoveItem(item, i, removeFactor)) {         return item;     } } return null;        }    }    public static class GuavaArrayListRemoveManyPerformer extends baseRemoveManyPerformer {        @Override        protected List<Integer> removeItems(List<Integer> items, final int removeFactor) { Iterables.removeIf(items, new Predicate<Integer>() {     public boolean apply(Integer input) {         return mustRemoveItem(input, input, removeFactor);     } }); return items;        }        @Override        protected List<Integer> createInitialList() { return new ArrayList<Integer>();        }    }    public void testForoneItemCnt(int itemCnt) {        testAll(itemCnt, 0);        testAll(itemCnt, itemCnt);        testAll(itemCnt, itemCnt - 1);        testAll(itemCnt, 3);        testAll(itemCnt, 2);        testAll(itemCnt, 1);    }    public static void main(String[] args) {        RemoveManyFromList t = new RemoveManyFromList();        t.addPerformer(new NaiveRemoveManyPerformer());        t.addPerformer(new BetterNaiveRemoveManyPerformer());        t.addPerformer(new linkedRemoveManyPerformer());        t.addPerformer(new CreateNewRemoveManyPerformer());        t.addPerformer(new SmartCreateNewRemoveManyPerformer());        t.addPerformer(new FasterSmartCreateNewRemoveManyPerformer());        t.addPerformer(new MagicRemoveManyPerformer());        t.addPerformer(new ForwardInPlaceRemoveManyPerformer());        t.addPerformer(new GuavaArrayListRemoveManyPerformer());        t.testForoneItemCnt(1000);        t.testForoneItemCnt(10000);        t.testForoneItemCnt(100000);        t.testForoneItemCnt(200000);        t.testForoneItemCnt(300000);        t.testForoneItemCnt(500000);        t.testForoneItemCnt(1000000);        t.testForoneItemCnt(10000000);    }}


转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/398338.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号