what is the most efficient way to move data across network? And st

Hi, I would to send data as the most efficient way, as compact and as fast as 
possible. So I think any serialization is out of picture.
A fundamental way is to translate all my fields in the class as bytes and 
pass it, then reconstruct it on other side. This may also save me a lot of 
hassle if passing between windows and linux, c# and c++.

So i try to use Buffer.BlockCopy to create byte array, pass it to socket. 
use BitConverter to read them back. I like to use fixed size array for 
efficient parsing at the client side. The only problem I run into is string

assuming I have one field called 
string ID; I know ID has variable lengh but less than 6. So I like to pass 
take 6 bytes of the array buffer and other side will take it and convert it 
back. The problem is that I cannot find a proper method in C# to do it. Seems 
I have to define the exact length to pass the array in C#.  is that possible 
to pass a variable length string to a fixed size byte array and convert it 
back as string?
thanks
0
Utf
2/19/2010 3:36:01 PM
dotnet.languages.csharp 1931 articles. 0 followers. Follow

5 Replies
913 Views

Similar Articles

[PageSpeed] 58

I'm gonna post this.........let me preface this as "some ideas"...........it 
might be off base for what your asking.............But here goes:


But check out this Serializer combined with this endpoint type if you're 
interested in WCF.

NetDataContractSerializer
"endpoint address=net.tcp"

I googled and found this example:
http://bertcraven.spaces.live.com/blog/cns!61EA3F98C8957A87!306.entry?sa=458189917


http://blogs.msdn.com/sowmy/archive/2006/06/06/all-about-knowntypes.aspx
This is a good WCF basic-understanding thing.

Keep in mind that you could write your own serializer.........you not tied 
to just the out of the box ones. 
(NetDataContractSerializer,DataContractSerializer, and the XmlSerializer)
http://www.lmgtfy.com/?q=wcf+custom+serialization

I've always used one of these with WCF.
NetDataContractSerializer
DataContractSerializer

But if you want small, where you can share the types, and size of the 
serialization is a priority, then NetDataContractSerializer combined with 
tcp endpoints.

This might be of small interest as well:
http://www.pluralsight.com/community/blogs/aaron/archive/2008/05/13/50934.aspx







"chrisben" <chrisben@discussions.microsoft.com> wrote in message 
news:E1466090-180B-43D3-AF0B-78B03E2743AE@microsoft.com...
> Hi, I would to send data as the most efficient way, as compact and as fast 
> as
> possible. So I think any serialization is out of picture.
> A fundamental way is to translate all my fields in the class as bytes and
> pass it, then reconstruct it on other side. This may also save me a lot of
> hassle if passing between windows and linux, c# and c++.
>
> So i try to use Buffer.BlockCopy to create byte array, pass it to socket.
> use BitConverter to read them back. I like to use fixed size array for
> efficient parsing at the client side. The only problem I run into is 
> string
>
> assuming I have one field called
> string ID; I know ID has variable lengh but less than 6. So I like to pass
> take 6 bytes of the array buffer and other side will take it and convert 
> it
> back. The problem is that I cannot find a proper method in C# to do it. 
> Seems
> I have to define the exact length to pass the array in C#.  is that 
> possible
> to pass a variable length string to a fixed size byte array and convert it
> back as string?
> thanks 


0
sloan
2/19/2010 3:54:30 PM
"chrisben" <chrisben@discussions.microsoft.com> wrote in message 
news:E1466090-180B-43D3-AF0B-78B03E2743AE@microsoft.com...
> Hi, I would to send data as the most efficient way, as compact and as fast 
> as
> possible. So I think any serialization is out of picture.
> A fundamental way is to translate all my fields in the class as bytes and
> pass it, then reconstruct it on other side. This may also save me a lot of
> hassle if passing between windows and linux, c# and c++.

Maybe json with numerics rather than field names.
With soap or http you can pass pretty much what you like.

WCF and tcp are efficient and you can avoid serializing except I'm not sure 
you can handle the data in whatever linuz process you have intimated might 
possibly be involved somehow. 

0
Andy
2/19/2010 4:45:41 PM
chrisben wrote:
> Hi, I would to send data as the most efficient way, as compact and as fast as 
> possible. So I think any serialization is out of picture.
> A fundamental way is to translate all my fields in the class as bytes and 
> pass it, then reconstruct it on other side. This may also save me a lot of 
> hassle if passing between windows and linux, c# and c++.

You are correct.  For efficiency of _transmission_ as well as 
interoperability, you can't beat conversion of data types to raw bytes 
according to a predefined, compact protocol and transmitting in that form.

> So i try to use Buffer.BlockCopy to create byte array, pass it to socket. 
> use BitConverter to read them back. I like to use fixed size array for 
> efficient parsing at the client side. The only problem I run into is string
> 
> assuming I have one field called 
> string ID; I know ID has variable lengh but less than 6. So I like to pass 
> take 6 bytes of the array buffer and other side will take it and convert it 
> back. The problem is that I cannot find a proper method in C# to do it. Seems 
> I have to define the exact length to pass the array in C#.  is that possible 
> to pass a variable length string to a fixed size byte array and convert it 
> back as string?

The first thing you need to do is understand character encoding. 
Characters are not bytes in C#.  They are text encoded as UTF-16, so 
each character takes up 2 bytes (4 bytes for surrogate pairs).

You can use the System.Text.Encoding class to convert between string and 
byte arrays, using whatever encoding for the bytes you want.  For 
example, UTF-8 (which supports all Unicode characters, but can still 
encode the first 128 character values in a single byte like ASCII does), 
or ASCII (which is exactly one byte per character, but which can 
represent only a very small subset of the characters that are possible 
in .NET).

As far as dealing with the length of the string, you can use a 
fixed-sized buffer if you want, but note that you'll still need some way 
of terminating strings that don't fill the buffer (e.g. zero byte 
terminator), and then copying the fixed-sized buffer back to a 
correctly-sized buffer after receipt of the data, so that the Encoding 
class doesn't return (e.g.) a string with embedded nulls.

Pete
0
Peter
2/19/2010 6:11:03 PM
There it is:

//Quote//
Extend the XmlObjectSerializer to create your own serializer to serialize 
and deserialize objects.



System..::.Object
  System.Runtime.Serialization..::.XmlObjectSerializer
    System.Runtime.Serialization..::.DataContractSerializer
    System.Runtime.Serialization.Json..::.DataContractJsonSerializer
    System.Runtime.Serialization..::.NetDataContractSerializer


http://msdn.microsoft.com/en-us/library/system.runtime.serialization.xmlobjectserializer.aspx


So maybe you can write up that sweet custom serializer that PeterD and you 
brought up....and post the code!

..........




"sloan" <sloan@ipass.net> wrote in message 
news:O8TeVvXsKHA.5936@TK2MSFTNGP04.phx.gbl...
>
> I'm gonna post this.........let me preface this as "some 
> ideas"...........it might be off base for what your asking.............But 
> here goes:
>
>
> But check out this Serializer combined with this endpoint type if you're 
> interested in WCF.
>
> NetDataContractSerializer
> "endpoint address=net.tcp"
>
> I googled and found this example:
> http://bertcraven.spaces.live.com/blog/cns!61EA3F98C8957A87!306.entry?sa=458189917
>
>
> http://blogs.msdn.com/sowmy/archive/2006/06/06/all-about-knowntypes.aspx
> This is a good WCF basic-understanding thing.
>
> Keep in mind that you could write your own serializer.........you not tied 
> to just the out of the box ones. 
> (NetDataContractSerializer,DataContractSerializer, and the XmlSerializer)
> http://www.lmgtfy.com/?q=wcf+custom+serialization
>
> I've always used one of these with WCF.
> NetDataContractSerializer
> DataContractSerializer
>
> But if you want small, where you can share the types, and size of the 
> serialization is a priority, then NetDataContractSerializer combined with 
> tcp endpoints.
>
> This might be of small interest as well:
> http://www.pluralsight.com/community/blogs/aaron/archive/2008/05/13/50934.aspx
>
>
>
>
>
>
>
> "chrisben" <chrisben@discussions.microsoft.com> wrote in message 
> news:E1466090-180B-43D3-AF0B-78B03E2743AE@microsoft.com...
>> Hi, I would to send data as the most efficient way, as compact and as 
>> fast as
>> possible. So I think any serialization is out of picture.
>> A fundamental way is to translate all my fields in the class as bytes and
>> pass it, then reconstruct it on other side. This may also save me a lot 
>> of
>> hassle if passing between windows and linux, c# and c++.
>>
>> So i try to use Buffer.BlockCopy to create byte array, pass it to socket.
>> use BitConverter to read them back. I like to use fixed size array for
>> efficient parsing at the client side. The only problem I run into is 
>> string
>>
>> assuming I have one field called
>> string ID; I know ID has variable lengh but less than 6. So I like to 
>> pass
>> take 6 bytes of the array buffer and other side will take it and convert 
>> it
>> back. The problem is that I cannot find a proper method in C# to do it. 
>> Seems
>> I have to define the exact length to pass the array in C#.  is that 
>> possible
>> to pass a variable length string to a fixed size byte array and convert 
>> it
>> back as string?
>> thanks
>
> 


0
sloan
2/19/2010 8:05:17 PM
On Fri, 19 Feb 2010 07:36:01 -0800, chrisben
<chrisben@discussions.microsoft.com> wrote:

>Hi, I would to send data as the most efficient way, as compact and as fast as 
>possible. So I think any serialization is out of picture.
How much data?  If there is enough data it is sometimes faster to
compress (e.g. zip) the data, transmit the compressed data and
decompress at the receiving end.  Whether this will be faster for you
depends on the systems you are working with and the volume of data you
are transferring.

rossum

0
rossum
2/19/2010 11:11:31 PM
Reply:

Similar Artilces:

Data Point on Graph "39409 versus 11/23/07"
I have numerous graphs wherein the X Axis represents the DATE. When I scroll my mouse along the graph line (searching for data points), I often see the accompanying data as... Series 1 Point "39409" Value 209 instead of: Series 1 Point "22-Nov" Value 209 HOW CAN I AVOID THE REFERENCE ON THE DATA LINE FROM REFLECTING THE SERIAL NUMBER INSTEAD OF THE DATE ITSELF??? (such as "22-Nov".) Thanks, FLKulchar What versio...

Pull data (sumif?) from an external workbook
I have a workbook "report". Col A has contains products. The number of products in column A may vary but each product features only once. eg: TOTAL SALES FOR JANUARY ColA ColB ItemX 4 ItemY 5 ItemZ 4 Another workbook "sales" contains the individual sales of these items in one month. So Col A contains the products which may feature once or several times. Col B contains the Qty. eg: SALES FOR JANUARY ColA ColB ItemX 1 ItemX 3 ItemY 5 ItemZ 2 ItemZ 2 Both workbooks reside...

Publisher-How do I change calendars 04 to 05 without losing data?
I have a current document 12 calendars with data in each day. I am trying to change the calendars from 04 to 05 with losing the existing data. Is there a way the data can stay in the same spot and the dates change. I also want to keep the same layout I currently have. I have tried using calendar options but cannot get it to work. Any help would be appreciated. Manually... You can try this, Expand your existing calendar by 12 months. On the toolbar, edit, office clipboard, this will give you 24 copied items you can collect. Copy your data, go to the next year, place your cursor where you...

installing network service dll (vmnetbridge.dll) fails to copy into system32
Trying to install a network service vmnetbridge.dll fails with err msg. "An error occured while copying file vmnetbridge.dll. Cannot copy file to destination directory. c:\windows User is in local admin group. UAC is set to OFF. Driver is digitally signed by Vmware, inc. Please advice how to rectify. QA4Ever ...

Form does not show recordset's data
Hi everyone and thanks for reading I am trying to set an access form's recordset on an ADODB.Recordset that was fabricated from scratch. This means that I have appended the fields I want and then I added some records. Finally, I am setting the form's recordset property on this recordset but I get #Error in the form's controls. Offcourse, I confirm that the controlsource of each one of the controls matches the corresponding field values of the recordset. (If I set the form's recordset property on a recordset made from an existing source (table or query) the form works f...

move mailboxes #5
Hello. I have windows 2000 sp4 and exchange 2000 sp3 august hotfix rollup. i have seperate server dedicted for mailbox moves. That box has ESM 2000. i'd like to upgrade that box to ESM 2003 (apply sp's). Will i still be able to move mailboxes, although i have not run 2003 adprep? Thanks. -bkw Why aren't you using the move mailbox wizard via ADUC? Nue "bkw" <bkw@discussions.microsoft.com> wrote in message news:D84CC2B8-8626-4473-B096-1D9757A0DB21@microsoft.com... > Hello. > I have windows 2000 sp4 and exchange 2000 sp3 august hotfix rollup. > i have se...

How do I save data on a form to my database?
I am new to Access and was wondering how I would save a record from my forum to the database. I am using an unbound forum. Thanks adamskiii, Why have you created an "unbound" form to operate on a table? If you're new to Access, you've just made your project much more difficult than it needs to be. You'll need to create a button on your form that, when clicked, runs an Append query against your table, using the current values on the open form. -- hth Al Campagna Microsoft Access MVP 2007-2009 http://home.comcast.net/~cccso...

How can I tranfer my data to my.swf in MFC program.?
I have my.swf and my.exe (MFC). I want to send to my.swf an data of my.exe. But I don't know it. Please tell me some idea.. (I want just simple flash/MFC code.. just...) On Mar 15, 7:07 am, "keandi" <kea...@lycos.co.kr> wrote: > I have my.swf and my.exe (MFC). > > I want to send to my.swf an data of my.exe. > But I don't know it. > > Please tell me some idea.. > > (I want just simple flash/MFC code.. just...) Post this in a .swf ng instead. See if ShockWave Flash object has an object model that you can use to do this. --- Ajay This mig...

Sharing PST or Outlook Data Files
After posting a note saying how I exported the Outlook data from the desktop PC to a back up PST file then importing that into Outlook on my laptop, people posted a link to a program called "OutlookSync". I downloaded it and installed it on both computers. I then ran it on both computers to initialise the programs. I then ran "Sync" on the PC. It created a database file which I then copied to the directory where the sync prog is installed on the laptop. I ran it and "Sync'ed" it. It seems to be working. I presume that when I work on the laptop that if I ma...

Bulk import of data in HR module
We have the HR module integrated with Payroll but are grossly underutilizing it. One goal is to be able to import a summary .csv file from our health insurance provider in order to populate the HR tables with health ins choices, COBRA info, etc. Integration Manager doesn't seem to see the HR module as a valid destination. Any way to get this (and other) data into Great Plains? Thanks! Doug There are two ways: a. eConnect. This is a programmers tool, but does makes things quite easy as it handles all the business logic. A record gets imported or rejected b. Direct table import. ...

Microsoft Dynamics 4.0 Data Migration Manager Sign in Issue
Hi, OK I'm a Syatem Administrator on our companies CRM system. I have installed DMM as recommended on my PC and installed with SQL Express. The problem I have is that the DMM has installed correctly and when I try to sign in it comes up with the error of 'You must have System Administrator security role to use this tool.' I have!!. My colleague who is also a system administrator has installed successfully and has been able to sign in use the DMM. We are both on the same version of Windows (Windows 7 Enterprise) and exactly the same role permissions on the CRM. Has anyone see...

Extact GPS Data from JPG image #2
Hi All I want to extract Exif / GPS Information from the JPG Image . For that I am using Gode Gurus tinter. http://codeguru.earthweb.com/cpp/g-m/gdi/gdi/article.php/c3655/ In this all the tags for GPS are available . My problem is the when i see any image in it do not shows me GPS tags . // ************************************************************************* I want to know how can i extract GPS information . and also want to know can GDI+ is used to extract GPS information . // ************************************************************************* In code their is one line . m_...

macro to move from directory to directory
I am writing/recording a macro to make changes in 64 different spreadsheets in 64 different folders on a network. I am just learning vba. I don't know how to tell Excel to move through the 64 folders, test for the spreadsheet to be sure it's there, then make changes. Any suggestions? TIA Dave I would use a worksheet to help out. In A2:A65, I'd put the 64 folders to check. In B1, I'd put the filename to look for. Then something like this: Option Explicit Sub testme2() Dim myRng As Range Dim myCell As Range Dim myFileName As String Dim TestStr As String ...

A way to set up an email to be sent automatically each week
Is there a way in Outlook 2003 to set up an email to be sent automatically each week? I wasn't sure if there was a way to set up a recurring email, weekly? Thanks for your time. "rrupp" <rrupp@discussions.microsoft.com> wrote in message news:1D41DADE-11F9-41BB-962C-3452959F3135@microsoft.com... > Is there a way in Outlook 2003 to set up an email to be sent automatically > each week? I wasn't sure if there was a way to set up a recurring email, > weekly? > > Thanks for your time. > Not without code. Please try our email sche...

Help with importing data
Hi All, I'm currently using Access 2002 Sp3 and I'm having troubles with importing data from a text file. The text file is 750Mb in size. Are there any handy tips or tricks that I ould using tryiong to import this data. Whenever I try to import data Access stops responding. Rgds, Dolphy With a file that size you have to do a line by line import ie write your own routine Pieter "Dolphy" <Kyrpto@gmail.com> wrote in message news:1190847728.817529.138530@g4g2000hsf.googlegroups.com... > Hi All, > > I'm currently using Access 2002 Sp3 and I'm having...

Monthly Summary of Data
I am having Daily Sales Figures for the entire year being (Invoice No., Date, Name of Seller, Net Sales Amount, Tax, Gross Sales Amount). Now if i want a Monthly summary from the entire data of say ( Net Sales, Tax, Gross Sales) how do i get it in excel. Hi I would use a Pivot Table to summarise the data. Take a look at Debra Dalgliesh's site for some guidance on how to sue Pivot Tables http://www.contextures.com/tiptech.html -- Regards Roger Govier "proshail" <proshail@discussions.microsoft.com> wrote in message news:C51FE172-7891-468A-A751-E63BEB19CD3C@microsof...

Joining data points on a line chart
I have the following as an example set of data points to be plotted on either a line or scatter chart (doesn't matter which as I am having the same problem with both!) X Y1 Y2 1 2 1.8 2 4 3 6 4.5 4 8 5 10 9 6 12 I get a line drawn between the points for Y1, but because there is not a Y2 value for every point on the X-axis then I just get points and no line, despite a line being selected for in the options! Does anyone know how to get this line added on? Thanks In each empty cell enter =NA(); this will display as #N/A. The line on chart will be continuous. OR Click chart; use T...

Moving Outlook e-mails from windows 2000 to xp professioal
I just got a new computer with windows xp professional and have outlook 2000 on it. I have e-mails on my old computer in outlook 2000 and windows 2000. What folder do I copy the old e-mails into on my new computer. Thank you for your help. Jay The file you need to back up is your Personal Folders file (*.pst) It's where all the mail, calendar, contacts etc are stored. Take a look at these pages for info on Outlook data backup: http://www.slipstick.com/config/backup.htm -- Russ Valentine [MVP-Outlook] "Jay Levy" <anonymous@discussions.microsoft.com> wrote in message ...

How to save and restore all data
Hi ! I'm using MS Access 2003 and Windows XP On the one hand, there are regular users of our db. They only change the data in the tables. They don't change forms, queries, reports or other things. Once in a while I have to make a change to a form, a query or other non-table-data things. But in the meantime, the regular users may keep changing data in the tables. I would like to get the latest data into my new db version. Is there a way of copying just the tables to the new db version ? Is there a way of copying just the table data to the new db version ? Any helpful suggestions o...

When I change my data, my formulas don't update the answers,why?
I have a personal finance sheet that has basic formulas which I update monthly. However, if I change a data cell, the other cell which has a formula linked to it does not pull the new data cell information. The formula is correct. The only way to "fix" my problem is to retype the exact same formula and then it pulls the new data. Any advise? Tools>Options>Calculation, check Automatic -- Kind Regards, Niek Otten Microsoft MVP - Excel "ayprlthing" <ayprlthing@discussions.microsoft.com> wrote in message news:C2165718-AF86-4C47-9FA5-857F76EBFD62@micros...

Exporting Excel data to Photoshop or similar?
Hi My friend is an amateur archaeologist. He has made a geophysic sampling device, and he can download the raw data into an Exce spreadsheet. Now he would like to visualise the data. The Excel chart features ar inadequate, so he wants to export the Excel data into (plugin-enhanced) copy of Photoshop or something similar. This wil allow him to do contrast enhancement, edge detection, gaussia smoothing, etc. The fundamental question is - what is the best way to get the data ou of Excel and into Photoshop, or a similar program? TIA Pau -- Message posted from http://www.ExcelForum.com I...

you and another user are attempting to change the same data at the same time
Access 2007 front end, SQL 2005 backend, using rdc I'm working away enhancing the layout of a form, subform and the code in one module. I do have some fields where an update will trigger processing and updates of other fields and records in the subform. As long as I make sure I do a accmdsaverecord before the processing and a me.refresh after the processing - I do not see the error above. All will be going along swimmingly until the above message starts appearing after field updates where once there was no problem. If I revert to an earlier copy of my mdb and import t...

extracting numerical data
I have cells in my worksheet containing results from using ="text"&COUNTIF... For example, one cell contains "exc 9", because there are nine times "exc" is seen in the column. I want to know if I can extract the numerical data from these cells to use in pie charts. Excel 2002 SP-2. If the number you want to extract always has a leading space before it, you can use =0+TRIM(MID(A1,FIND(" ",A1),1024)) where A1 contains the string of interest. -- Regards, Tushar Mehta www.tushar-mehta.com Excel, PowerPoint, and VBA add-ins, tutorials Custom...

any way to combine registers so z-out contains all?
Is there any way to combine all registers so that one z- out would contain all data from days work on different registers? We would like to be able to use one actual cash register, but be able to sell from multiple registers or work stations. Is this possible? As far as I know and understand you must z out each individual register. We have 3 machines that can do the register but use 1 everyday. The other 2 machines we z out after they are used for sales. So one might have a batch from 4/30/04 to 7/1/04 if it wasn't used for sales during that period. -- Elizabeth M. "JackT&qu...

Form will not allow data entry 01-30-08
Today my database which has been working fine for 10 years quit working. I have tabbed controls of which some are subforms. All of the subforms work fine, but the tab-control pages will not allow any data entry and no error is given. I checked to make sure that the page properties and field properties are not locked. I can enter data directly into the backend of this database. Why would this happen to only tab controls and not subforms. I suspect this is a problem of the back end as prior front ends (that have not been used for months) do not allow the changes also. Any suggestions....