Why open-iscsi has 2x slower writes than Samba via 10G Ethernet?Slow iSCSI speeds between WinServer2016 and...

What is the adequate fee for a reveal operation?

Counting models satisfying a boolean formula

Why do passenger jet manufacturers design their planes with stall prevention systems?

How to pronounce "I ♥ Huckabees"?

What is the significance behind "40 days" that often appears in the Bible?

Equivalents to the present tense

What exactly is this small puffer fish doing and how did it manage to accomplish such a feat?

Meme-controlled people

Are ETF trackers fundamentally better than individual stocks?

What are substitutions for coconut in curry?

Why does energy conservation give me the wrong answer in this inelastic collision problem?

Simplify an interface for flexibly applying rules to periods of time

How could an airship be repaired midflight?

How to make healing in an exploration game interesting

Is "upgrade" the right word to use in this context?

Bach's Toccata and Fugue in D minor breaks the "no parallel octaves" rule?

How to terminate ping <dest> &

Adventure Game (text based) in C++

How do you talk to someone whose loved one is dying?

How to write cleanly even if my character uses expletive language?

Are relativity and doppler effect related?

How to explain that I do not want to visit a country due to personal safety concern?

What is "focus distance lower/upper" and how is it different from depth of field?

Have the tides ever turned twice on any open problem?



Why open-iscsi has 2x slower writes than Samba via 10G Ethernet?


Slow iSCSI speeds between WinServer2016 and Win10 over 10G Ethernet













0















On my local file server I have raid-6 on 7x HDD drives.



dd if=/dev/zero of=tempfile bs=1M count=2048 conv=fdatasync


Local speed test gives me 349 MB/s write speed.



Remote writes to Samba from SSD (>2Gb/s read speed) gives me 259 MB/s writes.
But remote writes to iSCSI drive (on Win10 iSCSI initiator) gives me mere 151 Mb/s writes.



raid6 config - 128K chunk size, stripe_cache_size = 8191. Write intent bitmap is on SSD (Samsung 860 PRO, 4096K bitmap chunk).



Array mounted with options: rw,noatime,nobarrier,commit=999,stripe=128,data=writeback



open-iscsi setup: target is based on a 4Tb file.



Any hints why iSCSI is slower than Samba on writes?
Any hints how to improve iSCSI writes speed?



I assume it has something to do with desire of open-iscsi to flush writes to disk after each operation, which increases write amplification on raid6 due to excessive parity rewrites. But I am not sure how to fix it. Speed it more important than safety of currently written data in case of power outage.



As a side note older ietd iSCSI target had the ability to enable write-back mode (using IOMode=wb), and sustained write speed was much faster. Unfortunately, it seems to be currently unmaintained.









share





























    0















    On my local file server I have raid-6 on 7x HDD drives.



    dd if=/dev/zero of=tempfile bs=1M count=2048 conv=fdatasync


    Local speed test gives me 349 MB/s write speed.



    Remote writes to Samba from SSD (>2Gb/s read speed) gives me 259 MB/s writes.
    But remote writes to iSCSI drive (on Win10 iSCSI initiator) gives me mere 151 Mb/s writes.



    raid6 config - 128K chunk size, stripe_cache_size = 8191. Write intent bitmap is on SSD (Samsung 860 PRO, 4096K bitmap chunk).



    Array mounted with options: rw,noatime,nobarrier,commit=999,stripe=128,data=writeback



    open-iscsi setup: target is based on a 4Tb file.



    Any hints why iSCSI is slower than Samba on writes?
    Any hints how to improve iSCSI writes speed?



    I assume it has something to do with desire of open-iscsi to flush writes to disk after each operation, which increases write amplification on raid6 due to excessive parity rewrites. But I am not sure how to fix it. Speed it more important than safety of currently written data in case of power outage.



    As a side note older ietd iSCSI target had the ability to enable write-back mode (using IOMode=wb), and sustained write speed was much faster. Unfortunately, it seems to be currently unmaintained.









    share



























      0












      0








      0








      On my local file server I have raid-6 on 7x HDD drives.



      dd if=/dev/zero of=tempfile bs=1M count=2048 conv=fdatasync


      Local speed test gives me 349 MB/s write speed.



      Remote writes to Samba from SSD (>2Gb/s read speed) gives me 259 MB/s writes.
      But remote writes to iSCSI drive (on Win10 iSCSI initiator) gives me mere 151 Mb/s writes.



      raid6 config - 128K chunk size, stripe_cache_size = 8191. Write intent bitmap is on SSD (Samsung 860 PRO, 4096K bitmap chunk).



      Array mounted with options: rw,noatime,nobarrier,commit=999,stripe=128,data=writeback



      open-iscsi setup: target is based on a 4Tb file.



      Any hints why iSCSI is slower than Samba on writes?
      Any hints how to improve iSCSI writes speed?



      I assume it has something to do with desire of open-iscsi to flush writes to disk after each operation, which increases write amplification on raid6 due to excessive parity rewrites. But I am not sure how to fix it. Speed it more important than safety of currently written data in case of power outage.



      As a side note older ietd iSCSI target had the ability to enable write-back mode (using IOMode=wb), and sustained write speed was much faster. Unfortunately, it seems to be currently unmaintained.









      share
















      On my local file server I have raid-6 on 7x HDD drives.



      dd if=/dev/zero of=tempfile bs=1M count=2048 conv=fdatasync


      Local speed test gives me 349 MB/s write speed.



      Remote writes to Samba from SSD (>2Gb/s read speed) gives me 259 MB/s writes.
      But remote writes to iSCSI drive (on Win10 iSCSI initiator) gives me mere 151 Mb/s writes.



      raid6 config - 128K chunk size, stripe_cache_size = 8191. Write intent bitmap is on SSD (Samsung 860 PRO, 4096K bitmap chunk).



      Array mounted with options: rw,noatime,nobarrier,commit=999,stripe=128,data=writeback



      open-iscsi setup: target is based on a 4Tb file.



      Any hints why iSCSI is slower than Samba on writes?
      Any hints how to improve iSCSI writes speed?



      I assume it has something to do with desire of open-iscsi to flush writes to disk after each operation, which increases write amplification on raid6 due to excessive parity rewrites. But I am not sure how to fix it. Speed it more important than safety of currently written data in case of power outage.



      As a side note older ietd iSCSI target had the ability to enable write-back mode (using IOMode=wb), and sustained write speed was much faster. Unfortunately, it seems to be currently unmaintained.







      iscsi software-raid mdadm raid5 open-iscsi





      share














      share












      share



      share








      edited 1 min ago







      BarsMonster

















      asked 6 mins ago









      BarsMonsterBarsMonster

      742921




      742921






















          0






          active

          oldest

          votes











          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "2"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f958623%2fwhy-open-iscsi-has-2x-slower-writes-than-samba-via-10g-ethernet%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Server Fault!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f958623%2fwhy-open-iscsi-has-2x-slower-writes-than-samba-via-10g-ethernet%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          As a Security Precaution, the user account has been locked The Next CEO of Stack OverflowMS...

          Список ссавців Італії Природоохоронні статуси | Список |...

          Українські прізвища Зміст Історичні відомості |...